-- Logs begin at Tue 2022-10-04 09:39:36 UTC, end at Tue 2022-10-04 10:46:45 UTC. -- Oct 04 10:05:55.483628 dv-cinder-860227-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 04 10:05:55.491137 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 04 10:05:55.493797 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Oct 4 10:05:55 2022] *** Oct 04 10:05:55.493797 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 04 10:05:55.493797 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 04 10:05:55.493797 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: nodename: dv-cinder-860227-1-iscsi Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: machine: x86_64 Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: clock source: unix Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: pcre jit disabled Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: detected number of CPU cores: 4 Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: current working directory: / Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: detected binary path: /usr/bin/uwsgi-core Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: your processes number limit is 47680 Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: your memory page size is 4096 bytes Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: detected max file descriptor number: 2048 Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: lock engine: pthread robust mutexes Oct 04 10:05:55.494076 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: thunder lock: enabled Oct 04 10:05:55.501865 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 04 10:05:55.501865 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 04 10:05:55.533813 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: Python main interpreter initialized at 0x55b125d5a9c0 Oct 04 10:05:55.533813 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: python threads support enabled Oct 04 10:05:55.533813 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: your server socket listen backlog is limited to 100 connections Oct 04 10:05:55.533813 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: your mercy for graceful operations on workers is 90 seconds Oct 04 10:05:55.534170 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: mapped 403077 bytes (393 KB) for 2 cores Oct 04 10:05:55.534315 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: *** Operational MODE: preforking *** Oct 04 10:05:55.534588 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: *** uWSGI is running in multiple interpreter mode *** Oct 04 10:05:55.534588 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: spawned uWSGI master process (pid: 81352) Oct 04 10:05:55.534782 dv-cinder-860227-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 04 10:05:55.538230 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: spawned uWSGI worker 1 (pid: 81361, cores: 1) Oct 04 10:05:55.538230 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: spawned uWSGI worker 2 (pid: 81362, cores: 1) Oct 04 10:05:55.538230 dv-cinder-860227-1-iscsi devstack@keystone.service[81352]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 04 10:05:59.808415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:05:59.808415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: from cryptography.utils import int_from_bytes Oct 04 10:05:59.809193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:05:59.809193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: from cryptography.utils import int_from_bytes Oct 04 10:05:59.809310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:05:59.809310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: from cryptography.utils import int_from_bytes Oct 04 10:05:59.809718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 04 10:05:59.809718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: from cryptography.utils import int_from_bytes Oct 04 10:05:59.858437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 10:05:59.858608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 04 10:05:59.858692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 10:05:59.858790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 04 10:05:59.858863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] command line args: [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 10:05:59.858937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] command line args: [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 04 10:05:59.859055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 10:05:59.859157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 04 10:05:59.859228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ================================================================================ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 10:05:59.859286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ================================================================================ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 04 10:05:59.859471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] admin_token = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.859563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] admin_token = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.859709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config_dir = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.859797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config_dir = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.859874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.859950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.860055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] config_source = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.860131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] config_source = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.860231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] debug = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.860320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] debug = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.860425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.861711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.861711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.861862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.861922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.861983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_config_append = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_config_append = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_dir = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_dir = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.862981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.863990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.864889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.865725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.866086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.866230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.866362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] publish_errors = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] publish_errors = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.867905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] standard_threads = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] standard_threads = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.868956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_journal = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_journal = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_json = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_json = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.869950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_stderr = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.870062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_stderr = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.870180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] use_syslog = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.870320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] use_syslog = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.870448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.870562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 04 10:05:59.870673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.870783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.870895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.871927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.872954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.873994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.874952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.875934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.876968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.877979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.878917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.879949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.880966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.881993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.882702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.882833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.882947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.883919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.884984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.885886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.886972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.887979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.888973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.889900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.890941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.891977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.892949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.893999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.894882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.895914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.external = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.external = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.896925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.password = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.password = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] auth.token = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] auth.token = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.897795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.898911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.899926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.900981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.901892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.902953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.903983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.904906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.905837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.906945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.907973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.908979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.909884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.910962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.911981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.912902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.913948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.914914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.915807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.916894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.917900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.918948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.919954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.920977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.921995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.922950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.923974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.924988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.925903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.926929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.927948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.928806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.929457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.929622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.929741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.929897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.930947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.931967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.932975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.933890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.934941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.935978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.936996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.937917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.938948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.939965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.940990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.941899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.942945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.943928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.944982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81362) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 04 10:05:59.945340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81361) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 04 10:05:59.973605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.973737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.973840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.973940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.974068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.974150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.983084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:05:59.983181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.011900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.012041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.012173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.012250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.065089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.065275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.113904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.114133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.114599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.114834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.269708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.269915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.270817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.294955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.295744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.307134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.307380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.320195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.320595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.320737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.320820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.320904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81362) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.321863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81361) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 04 10:06:00.338911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.339121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:06:00.339313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.339462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.339542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:06:00.339658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:06:00.339850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.340195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.340314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 04 10:06:00.340440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.340590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.340697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.340854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.340955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.341113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.341371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.350453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.350755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.351051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.351178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.351448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.351562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.352063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.352554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.358364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.361241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.361528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.361816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.362101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.362407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.362669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.362939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.363214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.363486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.363760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.364030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.372731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.375640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.375927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.376209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.376491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.376778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.377041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.377313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.377588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.377859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.378135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.378429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.387618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.389080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.389363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.394076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.394390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.394675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.401028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.401318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.401743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.403508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.403799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.406891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.407187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.407477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.407766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.408044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.408318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.408430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.408585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.408714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.408850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.409002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.415483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.415770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.421358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.421651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.421936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.422238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.422520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.422800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.423072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.423339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.424270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.424549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.428407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.432295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.432592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.436784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.439811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.439948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.440168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.443166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.443467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.443753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.444042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.444285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.444374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.445235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.448273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.448572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.452759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.455918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.459170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.459474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.459762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.460047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.460424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.460716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.460917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.461205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.461473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.461733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.468825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.470394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.471912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.472188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.474870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.475156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.475438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.475736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.476134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.476657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.476947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.477216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.477482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.484293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.485849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.487171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.487464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.487546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.487742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.487834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.488021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.488401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.488689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.488977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.490513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.490799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.491083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.491887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.492291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.503380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.503667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.504213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.504498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.504875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.504964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.505166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.505444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.508402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.511343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.512814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.514291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.514575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.514980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.521331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.521433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.522890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.524355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.525756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.526153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.526463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.526744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.527286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.528777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.530266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.530553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.530947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.534570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.534862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.535142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.537264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.538862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.541743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.541972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81362) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.542142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.542277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.542456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.542550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.542737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.542827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.543079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.543347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.543627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.543899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.544166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.544440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.544774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.545046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.545318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.545583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81362) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.550512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.550809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.551103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.558469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81361) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 04 10:06:00.558751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.559031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.559307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.559580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.559850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.560141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.560419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.560696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.560965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.561245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.561643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.561932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.562228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81361) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 04 10:06:00.579732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b125d5a9c0 pid: 81362 (default app) Oct 04 10:06:00.582986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaafc6-042a-4fc1-b42e-c09e7fb21863 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:00.583271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaafc6-042a-4fc1-b42e-c09e7fb21863 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:00.583542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbaafc6-042a-4fc1-b42e-c09e7fb21863 None None] PATH_INFO: `/v3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:00.588442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Tue Oct 4 10:06:00 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:00.596872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55b125d5a9c0 pid: 81361 (default app) Oct 04 10:06:07.351577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:07.352082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:07.352162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:07.353431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 2/2] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:07.361191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:07.361744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:07.361744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:07.431753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: INFO dbcounter [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] Registered counter for database keystone Oct 04 10:06:07.450780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG oslo_db.sqlalchemy.engines [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81361) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 10:06:07.456805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writer thread running {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 10:06:07.634956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG passlib.handlers.bcrypt [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81361) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 10:06:07.635339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG passlib.handlers.bcrypt [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81361) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 10:06:07.692834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:07.694492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:07.820346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1/3] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 462 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 04 10:06:08.271215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:08.346948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: INFO dbcounter [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] Registered counter for database keystone Oct 04 10:06:08.369152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG oslo_db.sqlalchemy.engines [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81362) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Oct 04 10:06:08.375816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writer thread running {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 04 10:06:08.530800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3da12b6-1efb-4147-befa-4c26b9f5116c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:08.535026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:08.536104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:08.536575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:08.536852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:08.549430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:08.616614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 10:06:08.616958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d56e0f-6952-4606-9802-dc8dff9df9d4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:08.630171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 3/4] 10.222.0.13 () {58 vars in 1324 bytes} [Tue Oct 4 10:06:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:06:08.636658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:08.658222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e4ce65-5fae-4010-8500-37a3dbc6c03b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:08.662872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:08.665830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:08.666143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:08.666445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] PATH_INFO: `/v3/projects/fe7a43104df8480199dd56c9a29fe00d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:08.667943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] RBAC: Authorizing `identity:get_project(project_id=fe7a43104df8480199dd56c9a29fe00d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:08.726186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 04 10:06:08.726557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e3d2ad-1c20-425c-8b08-6a87547ef18e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:08.732895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 2/5] 10.222.0.13 () {56 vars in 1207 bytes} [Tue Oct 4 10:06:08 2022] GET /identity/v3/projects/fe7a43104df8480199dd56c9a29fe00d => generated 344 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:10.882684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317058b-a070-4213-b8c2-6f0c835fa88d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:10.883208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317058b-a070-4213-b8c2-6f0c835fa88d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:10.883208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e317058b-a070-4213-b8c2-6f0c835fa88d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:10.887264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 4/6] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:10.892892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:10.893306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:10.893450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:10.913649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:10.915222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:10.922945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 3/7] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 04 10:06:11.195775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317058b-a070-4213-b8c2-6f0c835fa88d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:11.207782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e317058b-a070-4213-b8c2-6f0c835fa88d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:11.212818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3b68b-b8c0-4b51-b5bd-f9a1cd61c7dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:11.213413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b68b-b8c0-4b51-b5bd-f9a1cd61c7dc admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:11.213687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b68b-b8c0-4b51-b5bd-f9a1cd61c7dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:11.213956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b68b-b8c0-4b51-b5bd-f9a1cd61c7dc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:11.221766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3b68b-b8c0-4b51-b5bd-f9a1cd61c7dc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:11.223550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3b68b-b8c0-4b51-b5bd-f9a1cd61c7dc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:11.232651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 5/8] 10.222.0.13 () {58 vars in 1324 bytes} [Tue Oct 4 10:06:11 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:06:11.238003 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:11.247446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077557ca-f3a4-4180-af1a-bd7f9bae3d24 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:11.251775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a64947-8ff5-4f7a-a6ff-dc8e7af8844d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:11.252393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a64947-8ff5-4f7a-a6ff-dc8e7af8844d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:11.252671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a64947-8ff5-4f7a-a6ff-dc8e7af8844d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:11.252943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a64947-8ff5-4f7a-a6ff-dc8e7af8844d admin admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:11.254258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a64947-8ff5-4f7a-a6ff-dc8e7af8844d admin admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:11.256078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a64947-8ff5-4f7a-a6ff-dc8e7af8844d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:11.258302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 4/9] 10.222.0.13 () {56 vars in 1198 bytes} [Tue Oct 4 10:06:11 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:13.452369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55760-7328-4691-a638-7aeaa8b9eaf4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:13.452899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55760-7328-4691-a638-7aeaa8b9eaf4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:13.452899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb55760-7328-4691-a638-7aeaa8b9eaf4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:13.454251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 6/10] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:13.459303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:13.459753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:13.459890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:13.478316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:13.479525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:13.504137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03cc3-bd24-4040-ada2-1592dbef4d39 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:13.504414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03cc3-bd24-4040-ada2-1592dbef4d39 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:13.504688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03cc3-bd24-4040-ada2-1592dbef4d39 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:13.506071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 7/11] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:13.511262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:13.511540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:13.511810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:13.554471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG passlib.handlers.bcrypt [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81362) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 04 10:06:13.554860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG passlib.handlers.bcrypt [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81362) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 04 10:06:13.560704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 5/12] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:13.604075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:13.605666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:13.703337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:13.826525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:13.852010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:13.932698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9ac13-76a3-4216-a04c-91f8479d2318 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:13.965899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:13.966494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:13.966772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:13.967043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:13.973542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:13.979106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:13.987913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:13.988783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:13.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:13.995048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:14.076533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:14.078569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701190ec-d09c-4166-ae18-e81f0608719c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.080938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:14.098105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a123d8df-a496-4fd8-9e56-99e96ae9fb90 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.117185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.118042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.118362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.118642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.132402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.133014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.133286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.133559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.134260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:14.134569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.135964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.148656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 7/15] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:14.154398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:14.185971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.187523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.258881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8a85df-8281-4939-89eb-bdbf85c2c607 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.287283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 9/16] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 10:06:14.291337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.292048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.292334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.292601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.292986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:14.293760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.295228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.295719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:14.296782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 8/17] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:14 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:14.302176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:14.316784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:14.390180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f278541c-483d-4120-a01a-ebb2ff2d0184 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.400832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db348a3-6b79-43a7-bffc-0d6dc7eb33c0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.424388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.425231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.425553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.425780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.433387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.434011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.434315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.434589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.435697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.437151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.438739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:14.444680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.444971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:14.446129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:06:14.464698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:14.466303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:06:14.467170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:06:14.467170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:06:14.467170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:14.467170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:06:14.471990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:14.527694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:14.529152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:14.550289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1043814-adfd-48df-9772-44fa9303fe89 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.577873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38e914-e3a3-422f-9796-be65c35e4be0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:14.585221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.585921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.586222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.586503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.587230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.domain.id=default, user.id=789f896e36c14ef9a5f6c8970e9b3e4a) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:14.587542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.588976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed62e19-b1d0-43ae-a3a1-59f31446c39f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.595824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1322 bytes} [Tue Oct 4 10:06:14 2022] GET /identity/v3/role_assignments?user.id=789f896e36c14ef9a5f6c8970e9b3e4a&scope.domain.id=default&role.id=e8cffbbc9b164ec5b458727785767845 => generated 233 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:14.618425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:14.618425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:14.618425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:14.618425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:14.618834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:14.618834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:14.619090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8d0d-601b-4934-8613-3f3d355ecc39 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:14.639586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:16.942362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c305414-9650-4c3b-b5d4-e6bfa887bd39 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:16.942856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c305414-9650-4c3b-b5d4-e6bfa887bd39 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:16.942856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c305414-9650-4c3b-b5d4-e6bfa887bd39 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:16.944789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:16.949770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872785-5d4d-4ecf-a63d-9d7b209c1bb5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:16.950160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872785-5d4d-4ecf-a63d-9d7b209c1bb5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:16.950343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30872785-5d4d-4ecf-a63d-9d7b209c1bb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:16.976501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-30872785-5d4d-4ecf-a63d-9d7b209c1bb5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:16.977694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-30872785-5d4d-4ecf-a63d-9d7b209c1bb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:17.058247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:17.357211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5481-d46f-4772-b02a-7ecd9fe26d6f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:17.357877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5481-d46f-4772-b02a-7ecd9fe26d6f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:17.358387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078c5481-d46f-4772-b02a-7ecd9fe26d6f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:17.360202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:17.365951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb015672-0a41-4dc4-9142-71583c1c266f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:17.366701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb015672-0a41-4dc4-9142-71583c1c266f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:17.367195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb015672-0a41-4dc4-9142-71583c1c266f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:17.395168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eb015672-0a41-4dc4-9142-71583c1c266f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:17.396715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eb015672-0a41-4dc4-9142-71583c1c266f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:17.452259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:17.485875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:17.552142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5481-d46f-4772-b02a-7ecd9fe26d6f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:17.673073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b445a13-27e2-41d3-87eb-635235013b7a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:17.673411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b445a13-27e2-41d3-87eb-635235013b7a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:17.673630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b445a13-27e2-41d3-87eb-635235013b7a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:17.674994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 14/26] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:17.686690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:17.687269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:17.687739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:17.696222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078c5481-d46f-4772-b02a-7ecd9fe26d6f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:17.723184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:17.724674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:17.734796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:17.735736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:17.736240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:17.736707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:17.748672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:17.753845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:17.754684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:17.755424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:17.762457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 13/27] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:17.764314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:17.768778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:17.844665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 15/28] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:17.890540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2b558f-e1e1-4c6a-b453-30b1cf829c05 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:17.901094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:17.954276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:17.954940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:17.955435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:17.955856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:17.956733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:17.957186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:17.959024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:17.973136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 14/29] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:17.979742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:18.085344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:18.091041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01f306-6359-4953-8f40-2b6dd5d79405 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:18.099273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:18.143410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb2e476-a895-468b-b75c-36ee8f24667a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:18.149125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c562add6-5674-4c17-969a-2a7ee319bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:18.149990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562add6-5674-4c17-969a-2a7ee319bbae None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.150545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562add6-5674-4c17-969a-2a7ee319bbae None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.151027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c562add6-5674-4c17-969a-2a7ee319bbae None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.151897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562add6-5674-4c17-969a-2a7ee319bbae None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:18.153589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c562add6-5674-4c17-969a-2a7ee319bbae None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:18.196521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:18.196521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.196521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.196521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.200988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:18.202549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:18.204163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 15/30] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:17 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:18.215624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:18.360411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b341c5-2d06-429a-b282-b867e27d919f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:18.371251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 16/31] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Oct 4 10:06:17 2022] POST /identity/v3/roles => generated 219 bytes in 470 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:06:18.373172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074377-c871-44d2-9972-124b11d48d31 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.373711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074377-c871-44d2-9972-124b11d48d31 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.374191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49074377-c871-44d2-9972-124b11d48d31 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.376161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 17/32] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:18.382593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.383186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.383709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.400801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:18.401375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.401654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.401925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.403180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:18.404688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:18.406002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:18.406401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 16/33] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:18.408067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:18.411931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:18.415838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:18.530534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019ca163-93e4-4b15-982f-bf2dc86ae969 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:18.548029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 18/34] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:18.549314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:18.573957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:18.574727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.575012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.575294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] PATH_INFO: `/v3/domains/default/users/789f896e36c14ef9a5f6c8970e9b3e4a/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.580114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=789f896e36c14ef9a5f6c8970e9b3e4a, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:18.584132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:18.724861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 17/35] 10.222.0.13 () {58 vars in 1361 bytes} [Tue Oct 4 10:06:18 2022] PUT /identity/v3/domains/default/users/789f896e36c14ef9a5f6c8970e9b3e4a/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:06:18.727681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:18.733640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77e24a-537f-4c84-aff0-d49605fac726 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:18.777584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:18.778628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.779078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.779493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.793350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:18.797382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-5b0c5c5c-eeeb-4ff4-8585-782a7ed0bc6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:06:18.815996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:18.817160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:06:18.818193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:06:18.818193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:06:18.818193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:18.821374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 19/36] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:18.827594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:18.937896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5763bbe-a000-4d4f-ba09-96a8e42559c0 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.938584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5763bbe-a000-4d4f-ba09-96a8e42559c0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.939127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5763bbe-a000-4d4f-ba09-96a8e42559c0 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:18.940768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 20/37] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:18.943063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:18.965961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:18.967038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:18.967821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.007059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:19.007928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:19.011887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6d46e9-b87b-4624-8114-539f01a53c6d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:19.053859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:19.055135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.055135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.055135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.055791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:19.056104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:19.058120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:19.112802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 21/38] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:19.119825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:19.190281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 18/39] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 364 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:19.196648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:19.247007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:19.278044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c163bb5-1130-431a-a56e-5dc6b2e52f1f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:19.320659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee057adf-18ec-4cfa-913a-6c19e3327f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:19.322038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee057adf-18ec-4cfa-913a-6c19e3327f60 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.322499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee057adf-18ec-4cfa-913a-6c19e3327f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.322939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee057adf-18ec-4cfa-913a-6c19e3327f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.323742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee057adf-18ec-4cfa-913a-6c19e3327f60 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:19.324235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0aca37c-9b00-4995-8bc1-ccfa702d52f9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:19.325421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee057adf-18ec-4cfa-913a-6c19e3327f60 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:19.341899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 22/40] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Oct 4 10:06:19 2022] POST /identity/v3/roles => generated 225 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:19.344387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00045b-eb66-4933-85b8-5c869a9597d1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.344897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00045b-eb66-4933-85b8-5c869a9597d1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.345410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00045b-eb66-4933-85b8-5c869a9597d1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.348115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 23/41] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:19.350571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:19.354349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.355110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.356208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.364538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:19.365218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.366573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.367053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.367986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:19.369849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:19.378646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:19.379869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:19.409365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 19/42] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:06:19 2022] POST /identity/v3/projects => generated 301 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:19.445227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:19.525822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 24/43] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:19.582584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19c1617-e1b5-4eb6-84c5-098e436ea927 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:19.643581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:19.644225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.644512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.644793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.645489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:19.648137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:19.669216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 20/44] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Oct 4 10:06:19 2022] POST /identity/v3/roles => generated 223 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:19.855823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:19.930821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:19.965084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd9b1ea-dc07-43e3-9f32-9300454679a3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:19.997348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af878d-5c56-4f75-8aa6-15374538659b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:19.997975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af878d-5c56-4f75-8aa6-15374538659b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:19.998281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af878d-5c56-4f75-8aa6-15374538659b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:19.998558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af878d-5c56-4f75-8aa6-15374538659b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:19.999887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af878d-5c56-4f75-8aa6-15374538659b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:20.001328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af878d-5c56-4f75-8aa6-15374538659b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:20.003197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:20.009650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:20.042601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:20.114154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74171cf-34e6-4a65-98a0-d20263f66edd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:20.149509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b7c923-7378-4e33-adec-1b8df258652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:20.150122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7c923-7378-4e33-adec-1b8df258652a None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:20.150437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7c923-7378-4e33-adec-1b8df258652a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:20.150722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b7c923-7378-4e33-adec-1b8df258652a None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:20.151431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7c923-7378-4e33-adec-1b8df258652a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:20.153040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b7c923-7378-4e33-adec-1b8df258652a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:20.180301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 21/46] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:06:20 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:06:21.195954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf47d8-9470-4d96-9f46-6a2c777bf523 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:21.196465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf47d8-9470-4d96-9f46-6a2c777bf523 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:21.196570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddf47d8-9470-4d96-9f46-6a2c777bf523 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:21.197825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:21.203127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:21.203518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:21.203682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:21.222335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:21.223831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:21.286433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:21.550914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf47d8-9470-4d96-9f46-6a2c777bf523 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:21.657551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddf47d8-9470-4d96-9f46-6a2c777bf523 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:21.690773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:21.691398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:21.691673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:21.691944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:21.696164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:21.704541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:21.710857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:21.711830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:21.712752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 27/49] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:21.719839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:21.737135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:21.816902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b707b6a7-6b3e-4d5a-9981-1ee4bc3dae26 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:21.849045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:21.850149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:21.850460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:21.850738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:21.851393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:21.851698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:21.853067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:21.860920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 23/50] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:21.867028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:21.968617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409ecc0-4a8c-47ec-96f5-e6ad0e097d10 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:22.119465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:22.120105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.120387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.120668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.122027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:22.123507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:22.125083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 28/51] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:21 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:22.130568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:22.207872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:22.231592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0536f7-28d7-44aa-9cd9-49ab3c82f8d4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:22.267980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c244b393-f194-4e8a-8c59-287c9e0422bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:22.268625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244b393-f194-4e8a-8c59-287c9e0422bf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.268906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244b393-f194-4e8a-8c59-287c9e0422bf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.269182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c244b393-f194-4e8a-8c59-287c9e0422bf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.270331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244b393-f194-4e8a-8c59-287c9e0422bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:22.272302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c244b393-f194-4e8a-8c59-287c9e0422bf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:22.273885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 24/52] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:22.279641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:22.341376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:22.353793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f0b02-29a7-4890-9d39-c1002c1a3937 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.354078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f0b02-29a7-4890-9d39-c1002c1a3937 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.354390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f0b02-29a7-4890-9d39-c1002c1a3937 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.355748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 25/53] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:22.360722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.361006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.361288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.379591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:22.380730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:22.383223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ac573b-d2f0-4b93-a638-8f45bcf90df8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:22.417036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:22.417869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.418145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.418456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.419170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.domain.id=default, user.id=789f896e36c14ef9a5f6c8970e9b3e4a) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:22.419478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:22.420941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:22.429595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 29/54] 10.222.0.13 () {56 vars in 1322 bytes} [Tue Oct 4 10:06:22 2022] GET /identity/v3/role_assignments?user.id=789f896e36c14ef9a5f6c8970e9b3e4a&scope.domain.id=default&role.id=e8cffbbc9b164ec5b458727785767845 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:22.450417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 26/55] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:22.724625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:22.819682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db2f9d2-8249-4293-b284-32ca022c2edc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:22.845245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:22.853184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd32cd4f-e58c-48da-9a9e-b56e184a2598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:22.853802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32cd4f-e58c-48da-9a9e-b56e184a2598 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.854087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32cd4f-e58c-48da-9a9e-b56e184a2598 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.854387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd32cd4f-e58c-48da-9a9e-b56e184a2598 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.855484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32cd4f-e58c-48da-9a9e-b56e184a2598 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:22.856917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd32cd4f-e58c-48da-9a9e-b56e184a2598 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:22.858491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 30/56] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:22.863968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:22.936889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:22.959680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da20405f-3554-49b1-98f0-6e5217e58311 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:22.992478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13c218-5adb-4b3f-9c98-ab1c13ad4593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:22.993069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c218-5adb-4b3f-9c98-ab1c13ad4593 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:22.993344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c218-5adb-4b3f-9c98-ab1c13ad4593 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:22.993612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13c218-5adb-4b3f-9c98-ab1c13ad4593 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:22.994272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13c218-5adb-4b3f-9c98-ab1c13ad4593 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:22.995720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13c218-5adb-4b3f-9c98-ab1c13ad4593 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:23.020163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 27/57] 10.222.0.13 () {60 vars in 1140 bytes} [Tue Oct 4 10:06:22 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:25.179709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7cf7-8e4a-442c-8acf-f2b392246e93 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:25.180215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7cf7-8e4a-442c-8acf-f2b392246e93 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:25.180301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6b7cf7-8e4a-442c-8acf-f2b392246e93 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:25.181579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 31/58] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:25.186664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:25.187091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:25.187219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:25.206474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:25.207663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:25.271960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 28/59] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:25.536566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b7cf7-8e4a-442c-8acf-f2b392246e93 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:25.636618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6b7cf7-8e4a-442c-8acf-f2b392246e93 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:25.669482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424fc18-3c65-4396-a69a-115d5ca57f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:25.670111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424fc18-3c65-4396-a69a-115d5ca57f69 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:25.670417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424fc18-3c65-4396-a69a-115d5ca57f69 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:25.670691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424fc18-3c65-4396-a69a-115d5ca57f69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:25.671794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424fc18-3c65-4396-a69a-115d5ca57f69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:25.673232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424fc18-3c65-4396-a69a-115d5ca57f69 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:25.674817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 32/60] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:25.681094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:25.735846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:25.776627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b868ba-15b2-4fdc-87af-2c29864eda62 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:25.809737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07da799-1d23-4fc9-9ab3-5362c924f57d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:25.810296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da799-1d23-4fc9-9ab3-5362c924f57d None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:25.810575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da799-1d23-4fc9-9ab3-5362c924f57d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:25.810848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07da799-1d23-4fc9-9ab3-5362c924f57d None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:25.811498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07da799-1d23-4fc9-9ab3-5362c924f57d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:25.812948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07da799-1d23-4fc9-9ab3-5362c924f57d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:25.838756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 29/61] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:25 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:29.096403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cecb88-66d9-4e0f-8add-dcc7aea78017 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.096877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cecb88-66d9-4e0f-8add-dcc7aea78017 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.096877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cecb88-66d9-4e0f-8add-dcc7aea78017 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.098376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 33/62] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:29.111018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f96ceb-d245-45f1-807e-d12132a27a01 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.111447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f96ceb-d245-45f1-807e-d12132a27a01 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.111624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f96ceb-d245-45f1-807e-d12132a27a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.135319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e3f96ceb-d245-45f1-807e-d12132a27a01 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:29.136507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e3f96ceb-d245-45f1-807e-d12132a27a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:29.221644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 30/63] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:29.289202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ebfec-483d-4026-aa82-412e487a801d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.289525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ebfec-483d-4026-aa82-412e487a801d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.289743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119ebfec-483d-4026-aa82-412e487a801d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.291272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 34/64] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:29.297033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298b7b-e787-4d0c-af5c-aad1141a27e0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.297327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298b7b-e787-4d0c-af5c-aad1141a27e0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.297604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c298b7b-e787-4d0c-af5c-aad1141a27e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.323217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2c298b7b-e787-4d0c-af5c-aad1141a27e0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:29.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2c298b7b-e787-4d0c-af5c-aad1141a27e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:29.386132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:29.428743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 31/65] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:29.490509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b301-f9c8-4ed4-bb44-89849c9ddbf9 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.490857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b301-f9c8-4ed4-bb44-89849c9ddbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.491141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4b301-f9c8-4ed4-bb44-89849c9ddbf9 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.492630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 35/66] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:29.497651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b372-b0c1-4645-b22f-73b7b985d503 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.497946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b372-b0c1-4645-b22f-73b7b985d503 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.498317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c13b372-b0c1-4645-b22f-73b7b985d503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.517668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b301-f9c8-4ed4-bb44-89849c9ddbf9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:29.531715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3c13b372-b0c1-4645-b22f-73b7b985d503 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:29.532909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c13b372-b0c1-4645-b22f-73b7b985d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:29.597213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:29.620196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 32/67] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:29.621629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ffd4-e977-4dc2-819e-deff78defac8 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.621915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ffd4-e977-4dc2-819e-deff78defac8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.622189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a2ffd4-e977-4dc2-819e-deff78defac8 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.626241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 33/68] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:29.632133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.633407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.633980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.654907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:29.656440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:29.662216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4b301-f9c8-4ed4-bb44-89849c9ddbf9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:29.705962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:29.707088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.707559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.708037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.719401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:29.724932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:29.725748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:29.726534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:29.728782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 36/69] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:06:29.740565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:29.748888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 34/70] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:29.764354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:29.830563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:29.913403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54621fe4-3d92-4d3a-8394-55bffc2654c4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:29.942142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:29.956627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:29.957249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:29.957531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:29.957837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:29.959498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:29.960013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:29.961557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:29.965131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce5c241-f830-42d8-8a78-e7b671e6e791 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:29.977418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 37/71] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:29.978303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.009779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.010929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.011410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.011889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.024692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:30.029799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:30.030648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:30.031386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 35/72] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:06:30.032072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.109113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260ef7b-0de4-42a9-bd9b-0fa70993fdf1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.134977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.143087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92fecba-6fd3-44b3-a30b-50a5d0526cd7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.144799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.145445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.145719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.145990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.156463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:30.161326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:30.162771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:30.163546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 38/73] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:06:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:30.164690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.181207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.181839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.182116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.182420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.183551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.185012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.186639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 36/74] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.187394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.213203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.271562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d162afb3-26cc-487f-a479-6c9e4adf81ed None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.291883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc08bf2-ff62-46ec-b310-ea570497f6ff None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.305375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.306006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.306305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.306576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.316401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:30.321136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:30.322020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:30.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:30.325289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.325896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.326176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.326523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 39/75] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:30.326617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.327171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.327363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:30.327672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.329053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bba701-08cd-4e13-a75b-7ec16cf649e3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.337308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 37/76] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.338702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0bbe-a8e7-43e8-b3b5-e79f0bf027f6 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.338990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0bbe-a8e7-43e8-b3b5-e79f0bf027f6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.339268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f0bbe-a8e7-43e8-b3b5-e79f0bf027f6 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.340746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 38/77] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:30.341463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0bbe-a8e7-43e8-b3b5-e79f0bf027f6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.367993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.440475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e07b33-1b01-45e0-8d35-0c1dd6e38800 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.445057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f0bbe-a8e7-43e8-b3b5-e79f0bf027f6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.448186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.475417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.476079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.476368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.476642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.477309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:30.477616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.478427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dba0fe-93ca-446c-8d10-efc7a1e53e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.479027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dba0fe-93ca-446c-8d10-efc7a1e53e58 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.479112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.479303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dba0fe-93ca-446c-8d10-efc7a1e53e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.479573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dba0fe-93ca-446c-8d10-efc7a1e53e58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.487032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 40/78] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.487823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.545734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dba0fe-93ca-446c-8d10-efc7a1e53e58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.547241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dba0fe-93ca-446c-8d10-efc7a1e53e58 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.572376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.586560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693f8c46-3c03-47d7-87f8-4b7be029a805 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.622556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.623174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.623462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.623742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.624408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:30.624711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.626100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.636513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 41/79] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.637367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.650676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 39/80] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:30.651918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.652207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.652478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.670906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:30.672096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:30.672421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.740712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 40/81] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:30.741476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.741625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9793bab-c9de-45a1-9962-d62292311da6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.777954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.778616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.778898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.779175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.795623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.797082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.803408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 42/82] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.803408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.810469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.857115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9136eb3-6676-4ea5-a197-827854f58349 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.891689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.892321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.892603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.892877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.893613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:30.894438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.897530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.903938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 41/83] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.903938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:30.915607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84fb64-3e2e-44cc-8278-31ca7c0b28d9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:30.951790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:30.952429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:30.952713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:30.952988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:30.954125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:30.955605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:30.957528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 43/84] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:30.958332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.014532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f80d8-2e88-4cd6-b95d-db9e2605c53f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.035673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.049973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.050628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.050906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.051179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.052308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.054424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.056049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 42/85] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:31.056807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.066558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f02e77-fd81-4f86-b2a5-a22e4d4fa357 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.098871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.100008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.100283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.100552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.125356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.154430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ac5f6c-dfe7-422e-8a35-43caaeffa866 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.168590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.170055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.187491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.188096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.188379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.188652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.244465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.255152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.256849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.267783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 44/86] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:31.268563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.321664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.356331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 43/87] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:31.357172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.366295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ad5f15-0d6f-4a1e-880c-b3b236e8953a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.400263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.400898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.401336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.401619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.445262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.455021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07329a-88dc-468d-9d86-6769c8a8ed2e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.469786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.471301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.488612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.489233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.489511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.490436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.491623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.493074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.494700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 44/88] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:31.495463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.524861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.569652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 45/89] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:31.570528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.594320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc5c713-81d0-4272-8307-fd180f4ef376 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.630055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.631192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.631907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.631907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.632553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=789f896e36c14ef9a5f6c8970e9b3e4a) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:31.632871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.634346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.640152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 45/90] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/role_assignments?user.id=789f896e36c14ef9a5f6c8970e9b3e4a&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:31.640941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.671534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.675105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabed2ab-ad59-47b9-ae4c-8e9563e22f10 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.717428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.718227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.718515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.718972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.720144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.721635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.723653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 46/91] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:31.725140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.755410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.765683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84408eb-b8c0-4f2e-898d-eedaa7697979 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.804150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.804880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.805230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.805508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.806694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.808140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.814764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 46/92] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:31.814764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.844021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba2a952-843d-4584-923e-b37a7e642ac2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.879633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.880248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.880579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.880857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.881511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.882973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.906060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:31.913652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 47/93] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:31 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:31.914526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:31.919956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb7f904-d9df-45d0-8057-b02419c3a390 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:31.973989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:31.975686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:31.976239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:31.976781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] PATH_INFO: `/v3/projects/34f80a261b4141b79ce3bae0df60bf5b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:31.979139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] RBAC: Authorizing `identity:get_project(project_id=34f80a261b4141b79ce3bae0df60bf5b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:31.981611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:31.984639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 47/94] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/projects/34f80a261b4141b79ce3bae0df60bf5b => generated 312 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:31.986551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:32.012191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:32.037697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd3bcbb-2906-429f-9b67-6a4dea743631 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:32.079934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:32.080544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:32.080826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:32.081106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:32.081899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=e55c08d9c6404546b2c05b48636468bc) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:32.082228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:32.084327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:32.091648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 48/95] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/role_assignments?user.id=e55c08d9c6404546b2c05b48636468bc&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=7c12d43981d3489c96be96ccaf741270 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:32.100621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:32.138664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a19f01-c4fd-403f-a038-b2125d414d73 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:32.189963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:32.192128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:32.192128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:32.192128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:32.192128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] RBAC: Adding query filter params (role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=e55c08d9c6404546b2c05b48636468bc) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:32.192381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:32.195307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1010280f-ea53-43d0-a1f3-7fb57379b0d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:32.204877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 48/96] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:31 2022] GET /identity/v3/role_assignments?user.id=e55c08d9c6404546b2c05b48636468bc&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:32.248096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67465d82-b2f5-4b7e-977f-f25a9d174097 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:32.294007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:32.295063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:32.295666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:32.296151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:32.297140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=34f80a261b4141b79ce3bae0df60bf5b, user.id=e55c08d9c6404546b2c05b48636468bc) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:32.297676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:32.299456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567364d6-9624-4c7b-b752-20cb3f830705 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:32.307354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 49/97] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:32 2022] GET /identity/v3/role_assignments?user.id=e55c08d9c6404546b2c05b48636468bc&scope.project.id=34f80a261b4141b79ce3bae0df60bf5b&role.id=7c12d43981d3489c96be96ccaf741270 => generated 259 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:34.328186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ddce4-c920-4ce4-b099-819d4a79c12e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:34.328689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ddce4-c920-4ce4-b099-819d4a79c12e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:34.328790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514ddce4-c920-4ce4-b099-819d4a79c12e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:34.330242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 49/98] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:34.335934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644a91-8fe2-451f-b8ab-e0deaa0f8796 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:34.336734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644a91-8fe2-451f-b8ab-e0deaa0f8796 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:34.337210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6644a91-8fe2-451f-b8ab-e0deaa0f8796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:34.358221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b6644a91-8fe2-451f-b8ab-e0deaa0f8796 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:34.359690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b6644a91-8fe2-451f-b8ab-e0deaa0f8796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:34.366191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:34.446224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 50/99] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:34.954670 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdf884-bd52-4d80-b388-de644fe6b9eb None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:34.955019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdf884-bd52-4d80-b388-de644fe6b9eb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:34.955190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cdf884-bd52-4d80-b388-de644fe6b9eb None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:34.956499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 50/100] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:34.962491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:34.962777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:34.963052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:34.980063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdf884-bd52-4d80-b388-de644fe6b9eb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:34.991109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:34.992250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:35.016931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:35.088570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 51/101] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:35.167457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cdf884-bd52-4d80-b388-de644fe6b9eb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:35.221765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2550fc21-a60f-418a-8483-f496012638e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:35.222346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550fc21-a60f-418a-8483-f496012638e7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.222631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550fc21-a60f-418a-8483-f496012638e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.222907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2550fc21-a60f-418a-8483-f496012638e7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.239402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2550fc21-a60f-418a-8483-f496012638e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2550fc21-a60f-418a-8483-f496012638e7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:35.245092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:35.245942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:35.246719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 51/102] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:35.252178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:35.365643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:35.415049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec0ccd-6398-4441-a8ff-35cabbd276d8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:35.522435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859314-316f-4b46-8fba-9f9e0672c8b7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.522435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859314-316f-4b46-8fba-9f9e0672c8b7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.522435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7859314-316f-4b46-8fba-9f9e0672c8b7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.522435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 52/103] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:35.530247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8a2cb-1f22-4054-b845-1763357c2480 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.530247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8a2cb-1f22-4054-b845-1763357c2480 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.530247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d8a2cb-1f22-4054-b845-1763357c2480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.534879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:35.535503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.535780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.536050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.537342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:35.537342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:35.538955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480ee2ea-0429-487b-9097-ea4d3d33acf3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:35.554558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 52/104] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:35.556844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d7653-4eee-437d-ae41-c33b6c86f754 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.557135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d7653-4eee-437d-ae41-c33b6c86f754 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.557413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3d7653-4eee-437d-ae41-c33b6c86f754 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.558888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 53/105] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:35.564763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d7653-4eee-437d-ae41-c33b6c86f754 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:35.580148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-14d8a2cb-1f22-4054-b845-1763357c2480 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:35.581352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-14d8a2cb-1f22-4054-b845-1763357c2480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:35.685404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3d7653-4eee-437d-ae41-c33b6c86f754 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:35.698593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 53/106] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:35.699855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd1e0a-9c35-40f1-b2b6-02cc1219253b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.700141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd1e0a-9c35-40f1-b2b6-02cc1219253b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.700415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bd1e0a-9c35-40f1-b2b6-02cc1219253b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.717568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:35.723360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:35.724191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.724785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.725471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.726686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:35.728439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:35.734863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 54/107] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:35 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:35.735717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:35.743190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-21bd1e0a-9c35-40f1-b2b6-02cc1219253b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:35.744415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-21bd1e0a-9c35-40f1-b2b6-02cc1219253b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:35.764592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:35.846951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 54/108] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:35.846951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d15ce-114d-45ae-9844-e80d9e6e45ad None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:35.846951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d15ce-114d-45ae-9844-e80d9e6e45ad None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:35.846951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d15ce-114d-45ae-9844-e80d9e6e45ad None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:35.846951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 55/109] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:35.846951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d15ce-114d-45ae-9844-e80d9e6e45ad None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:35.950865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b21e6f-ea05-438a-8109-856efc8ceeb1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.022184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2038f2-4e87-41b9-9f13-59e91beb9d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.023187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2038f2-4e87-41b9-9f13-59e91beb9d0b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.023869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2038f2-4e87-41b9-9f13-59e91beb9d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.024469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2038f2-4e87-41b9-9f13-59e91beb9d0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.044262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2038f2-4e87-41b9-9f13-59e91beb9d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7c2038f2-4e87-41b9-9f13-59e91beb9d0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:36.054769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:36.055704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:36.056435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:36.060691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 55/110] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:36.063417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.064149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.064595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.070956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d15ce-114d-45ae-9844-e80d9e6e45ad None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.086961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:36.088141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:36.112956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ca9a1-cb34-4719-a16a-2b9df74c5aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.113767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca9a1-cb34-4719-a16a-2b9df74c5aee None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.114055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca9a1-cb34-4719-a16a-2b9df74c5aee None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.114374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ca9a1-cb34-4719-a16a-2b9df74c5aee None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.116686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ca9a1-cb34-4719-a16a-2b9df74c5aee None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:36.118147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ca9a1-cb34-4719-a16a-2b9df74c5aee None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:36.119872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 56/111] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:35 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:36.120983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929cec1-4a15-4dfd-8579-2e5ec91b5559 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.121266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929cec1-4a15-4dfd-8579-2e5ec91b5559 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.121541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929cec1-4a15-4dfd-8579-2e5ec91b5559 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.126973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 57/112] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:36.127917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929cec1-4a15-4dfd-8579-2e5ec91b5559 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.147594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.152157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.167797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 56/113] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:36.171102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.238441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929cec1-4a15-4dfd-8579-2e5ec91b5559 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.274509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.275161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.275439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.275716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.281267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42bec2b-9c2b-4f2b-a998-b672e87e1bf8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.285774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:36.291101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:36.291957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:36.292693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 58/114] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:36.293345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.319805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.320523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.320797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.321068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.321706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:36.322003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:36.323652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0189d42-84aa-40e4-a024-f832c5be74ef None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:36.333117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 57/115] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:36.334630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.334913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.335187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.355705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:36.357155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:36.365028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.397980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b8c80-6e57-49a6-8b4c-353bbfdee5ec None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.410878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.434100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 58/116] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:36.435085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.435750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.436027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.436293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da/users/789f896e36c14ef9a5f6c8970e9b3e4a/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.438191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] RBAC: Authorizing `identity:create_grant(project_id=6067280cdc4440acac78a21b0daba4da, user_id=789f896e36c14ef9a5f6c8970e9b3e4a, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:36.439662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:36.443779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.453927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 59/117] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:36 2022] PUT /identity/v3/projects/6067280cdc4440acac78a21b0daba4da/users/789f896e36c14ef9a5f6c8970e9b3e4a/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:36.455368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.566929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4d2317-c715-40fe-874f-213b01e3271e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.598791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c462b0c6-3b16-4086-8de4-1c2e3b6d8c66 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.615142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.618428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.618428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.618428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.630623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:36.636439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:36.637286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:36.638043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 59/118] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:36.638707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.655079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.666595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.667642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.668128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.668596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.669601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:36.670006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:36.671547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:36.685783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 60/119] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:36.685783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.697335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.759878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8206e54-90e6-43c6-bf3e-73756d7312b6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.794532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.795188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.795507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.795736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.796857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:36.798321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:36.799509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd78e0-5067-4d60-bc43-25b49f85e46b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.799891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 60/120] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:36.805091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.836183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.837073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.837499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.837768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.847966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:36.853680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:36.854817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 61/121] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:36.859131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.910565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.922637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b966d6-06fa-4188-9e5c-f860abb2c2ef None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.932296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:36.957370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39edeb5-adff-4fd4-8afe-429881bfc83c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:36.962053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f968-80af-471a-9926-5321a8260853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.962751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f968-80af-471a-9926-5321a8260853 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.963033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f968-80af-471a-9926-5321a8260853 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.963307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f968-80af-471a-9926-5321a8260853 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.963974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729f968-80af-471a-9926-5321a8260853 None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:36.964276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729f968-80af-471a-9926-5321a8260853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:36.966050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e729f968-80af-471a-9926-5321a8260853 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:36.979610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 61/122] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:36.979610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f968-80af-471a-9926-5321a8260853 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:36.996717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:36.997632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:36.997912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:36.998184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:36.999463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.001263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.008530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 62/123] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.008530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.112502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f968-80af-471a-9926-5321a8260853 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.124038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baddc45-97f0-4bba-a247-4903860997e6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.147466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:37.150891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.151534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.151879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.152160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.153711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.155931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.157527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 62/124] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.159409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.163085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.163686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.164657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.164657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.165435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.166915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.168739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 63/125] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.169342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.207281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:37.272673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ce5d6f-bb01-41c5-99ba-cb1f0c0c1c12 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.276406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b6b4d6-f16c-4734-9d49-05b29d53e017 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.308592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.309343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.309623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.309892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.310605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:37.310871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.311696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.312240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.312332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.312587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.312860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.313973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.315489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.321681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 64/126] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.321681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.325815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 63/127] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.327090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.378365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:37.430505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86710289-dd68-4ee9-a167-63f633fcd8ee None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.450724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:37.454889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9756bc9a-a0ce-4ee0-b547-8533944ab048 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.469974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.470642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.471086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.471363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.472613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.474895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.481298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 65/128] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.481861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.489182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.490426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.490426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.491014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.491640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.493078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.533816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 64/129] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:06:37 2022] POST /identity/v3/projects => generated 302 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:37.535084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.615510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c692581-c937-43d4-b1d5-4b876d44d33d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.654034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:37.668028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.669205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.669686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.670183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da/users/e55c08d9c6404546b2c05b48636468bc/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.670327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c103bc0b-d418-43ff-9741-fa3d3b41d2c5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.672624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] RBAC: Authorizing `identity:create_grant(project_id=6067280cdc4440acac78a21b0daba4da, user_id=e55c08d9c6404546b2c05b48636468bc, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.674459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.693192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 66/130] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:37 2022] PUT /identity/v3/projects/6067280cdc4440acac78a21b0daba4da/users/e55c08d9c6404546b2c05b48636468bc/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:37.694635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.726963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.727960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.728241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.728557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] PATH_INFO: `/v3/projects/34f80a261b4141b79ce3bae0df60bf5b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.730754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] RBAC: Authorizing `identity:get_project(project_id=34f80a261b4141b79ce3bae0df60bf5b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.732561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.734762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 65/131] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/projects/34f80a261b4141b79ce3bae0df60bf5b => generated 312 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.735424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.768503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:37.851638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013cdfd3-c539-4303-b0f3-022c8e2a9c6e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.888282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb977a4c-a8d9-42c9-a81b-b46847675359 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:37.899301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.900453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.900737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.901412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] PATH_INFO: `/v3/users/a5af0635781d459eb4cc38af780ab735` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.918761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] RBAC: Authorizing `identity:get_user(user_id=a5af0635781d459eb4cc38af780ab735)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.920812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.923398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 67/132] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/users/a5af0635781d459eb4cc38af780ab735 => generated 289 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:37.924169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.937162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:37.937847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:37.938124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:37.938438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da/users/e55c08d9c6404546b2c05b48636468bc/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:37.944543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] RBAC: Authorizing `identity:create_grant(project_id=6067280cdc4440acac78a21b0daba4da, user_id=e55c08d9c6404546b2c05b48636468bc, role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:37.946074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:37.968022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 66/133] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:37 2022] PUT /identity/v3/projects/6067280cdc4440acac78a21b0daba4da/users/e55c08d9c6404546b2c05b48636468bc/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:37.968811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:37.986378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:38.067774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69045ff-172e-4a6c-9edb-f164b1d88046 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:38.108488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:38.109567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:38.110080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:38.110575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] PATH_INFO: `/v3/projects/34f80a261b4141b79ce3bae0df60bf5b/users/e55c08d9c6404546b2c05b48636468bc/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:38.112737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] RBAC: Authorizing `identity:create_grant(project_id=34f80a261b4141b79ce3bae0df60bf5b, user_id=e55c08d9c6404546b2c05b48636468bc, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:38.115053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:38.138165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 68/134] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:37 2022] PUT /identity/v3/projects/34f80a261b4141b79ce3bae0df60bf5b/users/e55c08d9c6404546b2c05b48636468bc/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:38.154844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:38.184523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2afeeb-7c4a-4586-8e25-359ccd3934ef None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:38.232764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:38.233735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:38.234231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:38.234696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:38.325281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:38.326813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:38.431913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:38.454792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 67/135] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:38.461076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:38.591661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61543c81-3b57-43f8-a981-44fbcfa8cdc4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:38.593107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:38.649744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3bd5ab-c6a3-456b-90c8-2808f29e8078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:38.650501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bd5ab-c6a3-456b-90c8-2808f29e8078 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:38.650718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bd5ab-c6a3-456b-90c8-2808f29e8078 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:38.650993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3bd5ab-c6a3-456b-90c8-2808f29e8078 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:38.652151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bd5ab-c6a3-456b-90c8-2808f29e8078 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:38.653624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3bd5ab-c6a3-456b-90c8-2808f29e8078 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:38.655436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 69/136] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:38 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:38.661867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:38.786356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ea399-78e8-49b9-b51e-ffbdef81046d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:38.830037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:38.830757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:38.831103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:38.831442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:38.832244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] RBAC: Adding query filter params (role.id=7ef4aa4b4180492cb81a40024c482a71, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=a5af0635781d459eb4cc38af780ab735) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:38.832623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:38.835180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd17de3-d306-4325-bac1-e3eeffd41f5a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:38.846955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 68/137] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:38 2022] GET /identity/v3/role_assignments?user.id=a5af0635781d459eb4cc38af780ab735&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=7ef4aa4b4180492cb81a40024c482a71 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:39.316426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfed00b-825b-4297-8060-c597adad4564 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:39.316790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfed00b-825b-4297-8060-c597adad4564 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:39.316980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfed00b-825b-4297-8060-c597adad4564 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:39.318916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 70/138] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:39.324540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:39.324942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:39.325320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:39.348147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:39.349853 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:39.406346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:39.441157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 69/139] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:39.982409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfed00b-825b-4297-8060-c597adad4564 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:40.112595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfed00b-825b-4297-8060-c597adad4564 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:40.156099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:40.156748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.157208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.157535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.175113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:40.180820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:40.181620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:40.182384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 71/140] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:40.188220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:40.327512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6106907e-00fa-48bc-8993-ff1164cab4eb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:40.376903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:40.377524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.377805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.378083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.378777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:40.379082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:40.380560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:40.388775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 70/141] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:40.394676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:40.428358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:40.505919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720731b2-e796-42e9-81e2-37a5f9bd8448 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:40.549698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae053046-310f-407c-a9b8-cf000b63ce5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:40.550355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae053046-310f-407c-a9b8-cf000b63ce5f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.550633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae053046-310f-407c-a9b8-cf000b63ce5f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.550901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae053046-310f-407c-a9b8-cf000b63ce5f None admin] PATH_INFO: `/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.554920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae053046-310f-407c-a9b8-cf000b63ce5f None admin] RBAC: Authorizing `identity:get_user(user_id=789f896e36c14ef9a5f6c8970e9b3e4a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:40.556375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae053046-310f-407c-a9b8-cf000b63ce5f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:40.558406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 72/142] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:40 2022] GET /identity/v3/users/789f896e36c14ef9a5f6c8970e9b3e4a => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:40.564006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:40.604928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:40.622237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5aa9-dccb-4141-a6a2-73eb2a8bff61 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.622237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5aa9-dccb-4141-a6a2-73eb2a8bff61 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.622237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aa5aa9-dccb-4141-a6a2-73eb2a8bff61 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.622237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 73/143] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:40.628477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61f654-805d-4f6e-98d1-8c4761aef1a8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.628477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61f654-805d-4f6e-98d1-8c4761aef1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.628477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61f654-805d-4f6e-98d1-8c4761aef1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.647893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8d61f654-805d-4f6e-98d1-8c4761aef1a8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:40.649097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8d61f654-805d-4f6e-98d1-8c4761aef1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:40.700071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a585-7536-4427-838a-421d93b40581 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:40.742534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 74/144] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:40.744590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:40.745296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.745649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.745999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.815651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:40.817243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:40.843747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badecea2-45fa-411d-a6bd-8a1a9d10c500 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.844354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badecea2-45fa-411d-a6bd-8a1a9d10c500 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.844825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badecea2-45fa-411d-a6bd-8a1a9d10c500 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.846412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 75/145] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:40.855508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:40.856030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:40.856495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:40.862238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:40.891598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:40.893109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:40.934345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:40.990381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 71/146] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:40.998226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:41.002694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 76/147] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:41.093162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b565572-686b-452a-931f-57e3505ee266 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:41.134621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:41.135183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:41.135468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:41.135759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:41.136895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:41.138354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:41.140279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 72/148] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:40 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:41.146045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:41.223403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:41.262597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:41.276350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d828cfa-cb02-4403-8ce9-0d9c666e1783 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:41.343033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:41.344011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:41.344501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:41.345126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:41.346066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=789f896e36c14ef9a5f6c8970e9b3e4a) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:41.346591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:41.348259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:41.355574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 77/149] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:41 2022] GET /identity/v3/role_assignments?user.id=789f896e36c14ef9a5f6c8970e9b3e4a&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:41.413575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca3eadc-feb7-46fe-a936-59acc337d495 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:41.431339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:41.467001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:41.471809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:41.472441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:41.472723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:41.472997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:41.479086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:41.480594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:41.482553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 73/150] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:41.491256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:41.597172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff8d059-ae78-4de9-b639-971945e56a82 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:41.620362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:41.675661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6a1bbf-e6f7-4aa4-98c6-85ee7cf11d2b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:41.689619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:41.690291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:41.690572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:41.690843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:41.707462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:41.715490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:41.716319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:41.717136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 78/151] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:41.723024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:41.743768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:41.744733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:41.745228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:41.745695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:41.746692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:41.748336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:41.789120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 74/152] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:41 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:06:41.820977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:41.845386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad4e5bf-6876-47f4-813a-60158666a4bd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:41.889015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:41.889663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:41.889943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:41.890279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:41.890960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:41.891260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:41.892628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af5c03d-4666-496a-8317-578f09efb700 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:41.906490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 79/153] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:41.912902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.041900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74786b9-fc0a-4fca-8533-b3a5dc73baa2 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.043026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74786b9-fc0a-4fca-8533-b3a5dc73baa2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.043592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74786b9-fc0a-4fca-8533-b3a5dc73baa2 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.045147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 80/154] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:42.052112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86fe4-9fda-4e81-85d8-6d58a51fb7d1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.052698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86fe4-9fda-4e81-85d8-6d58a51fb7d1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.053157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad86fe4-9fda-4e81-85d8-6d58a51fb7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.053815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369180b6-ecfb-486f-82a0-d0a269aca601 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:42.084474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1ad86fe4-9fda-4e81-85d8-6d58a51fb7d1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:42.090514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1ad86fe4-9fda-4e81-85d8-6d58a51fb7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:42.091264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:42.101755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:42.102455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.102735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.103013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.104237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:42.107714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:42.108060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 75/155] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:41 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:42.119450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.212641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 81/156] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:42.215021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ff8e7-4067-4ada-96e6-79b40e3f24b4 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.215307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ff8e7-4067-4ada-96e6-79b40e3f24b4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.215578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8ff8e7-4067-4ada-96e6-79b40e3f24b4 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.216853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 82/157] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:42.225845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d3e44-0140-4cf0-b522-fbd513a9b306 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.226116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d3e44-0140-4cf0-b522-fbd513a9b306 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.226402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923d3e44-0140-4cf0-b522-fbd513a9b306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.247914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-923d3e44-0140-4cf0-b522-fbd513a9b306 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:42.249666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-923d3e44-0140-4cf0-b522-fbd513a9b306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:42.259397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a31b16-a383-4115-b8c6-35b3d80dab4a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:42.304998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:42.306610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.307203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.307647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.337793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 83/158] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:42.341515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbbfb4-b6a3-40b2-a183-cacd5251b72f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.341515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbbfb4-b6a3-40b2-a183-cacd5251b72f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.341515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdbbfb4-b6a3-40b2-a183-cacd5251b72f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.341515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 84/159] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:42.346696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.346978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.347252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.357250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:42.369973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:42.371478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:42.392170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:42.400059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:42.401642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:42.456117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 85/160] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:42.511579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 76/161] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:42.517103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.527820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.633707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53ebb9-262d-4d47-bf43-f0119348d54d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:42.636287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:42.643473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b603165-25f5-4653-a693-3b1311ad618b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:42.671094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:42.671728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.672001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.672266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.673372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:42.674833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:42.676455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 86/162] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:42.681248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:42.681891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.682445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.682723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.683103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.685730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:42.694822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:42.699978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:42.700839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:42.701637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 77/163] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:42.702883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.791976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65791f69-28aa-457d-901b-2722b774ac6d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:42.807776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79927eeb-4e23-4092-94a8-8b04ab102796 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:42.831606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:42.833196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.833662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.834572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.835754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=e55c08d9c6404546b2c05b48636468bc) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:42.836680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:42.838942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:42.847747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 87/164] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/role_assignments?user.id=e55c08d9c6404546b2c05b48636468bc&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=7c12d43981d3489c96be96ccaf741270 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:42.849086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:42.849708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.849860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:42.850110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:42.850414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:42.867461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:42.872771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:42.873694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:42.874522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 78/165] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:42.875412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:42.902865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:42.961083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:43.022282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dffe09-599c-4a06-9de7-9104c1299e8a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.034081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d651c4b-1fa3-43ef-b1f3-b2a7f2c50d52 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.079746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.080341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.080816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.080816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.081183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.081788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.082296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.082728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.084129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:43.085119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.086702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.094888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:43.097706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 88/166] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:43.099293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:43.102300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:43.103371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:43.104423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 79/167] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:43.108946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.202858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:43.229353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01ffa1-c368-413f-958e-8fb3d3cde545 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.268570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd2dd38-66e6-4dad-969c-9b88380bff07 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.285605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.287029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.287308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.287766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.288939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.290500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.293847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 80/168] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:43.294949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.307861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:43.324456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.325078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.325368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.325633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.326336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:43.326641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.328025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.341875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 89/169] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:43.346393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.452350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2e2825-8fd3-4ea8-bd42-a63b3bb5344b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.507116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0444afe6-9684-4480-a4fc-29e36c9f6891 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.518012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.518988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.519484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.519977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.520998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:43.521582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.523481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.543713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 81/170] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:43.545751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.566498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:43.579273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.580146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.580580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.581040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.704859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:43.720103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.721805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.764146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0cdca4-69a6-4643-b4db-5e48338a0cde None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.814660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49182c24-523d-40c1-b827-07543715d17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.815360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182c24-523d-40c1-b827-07543715d17f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.815895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182c24-523d-40c1-b827-07543715d17f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.816176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49182c24-523d-40c1-b827-07543715d17f None admin] PATH_INFO: `/v3/users/e55c08d9c6404546b2c05b48636468bc` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.817825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49182c24-523d-40c1-b827-07543715d17f None admin] RBAC: Authorizing `identity:get_user(user_id=e55c08d9c6404546b2c05b48636468bc)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.819305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49182c24-523d-40c1-b827-07543715d17f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.820961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 82/171] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/users/e55c08d9c6404546b2c05b48636468bc => generated 275 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:43.826740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49182c24-523d-40c1-b827-07543715d17f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.916854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:43.921603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 90/172] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 579 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:43.922546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:43.946509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49182c24-523d-40c1-b827-07543715d17f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:43.984596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:43.985570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:43.986068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:43.986802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] PATH_INFO: `/v3/users/a5af0635781d459eb4cc38af780ab735` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:43.988151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] RBAC: Authorizing `identity:get_user(user_id=a5af0635781d459eb4cc38af780ab735)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:43.990794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:43.993103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 83/173] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/users/a5af0635781d459eb4cc38af780ab735 => generated 289 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:43.994616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:44.104815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159b893b-b6b4-494c-97ae-ae9a840f7cf5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:44.152059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:44.166705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079c1f6c-030c-4483-9618-d9940859b644 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:44.182998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:44.184023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.184023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.184286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] PATH_INFO: `/v3/projects/34f80a261b4141b79ce3bae0df60bf5b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.187364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] RBAC: Authorizing `identity:get_project(project_id=34f80a261b4141b79ce3bae0df60bf5b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:44.189283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:44.191491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 84/174] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/projects/34f80a261b4141b79ce3bae0df60bf5b => generated 312 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:44.192392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:44.215016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:44.216271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.216906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.217400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.218576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:44.353503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:44.355078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:44.391766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2775d08d-01f5-4886-ba6c-b5a535646acc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:44.431749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:44.432736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.433207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.434529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.439355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:44.441164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:44.443607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 85/175] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:44 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:44.446568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:44.534549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:44.553017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 91/176] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 631 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:44.553763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:44.598240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:44.638223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f7b9a-6014-4674-921f-3d1c96260d53 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:44.667215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:44.667898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.668177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.668448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.669170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=34f80a261b4141b79ce3bae0df60bf5b, user.id=e55c08d9c6404546b2c05b48636468bc) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:44.669479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:44.670950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:44.677507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 86/177] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:44 2022] GET /identity/v3/role_assignments?user.id=e55c08d9c6404546b2c05b48636468bc&scope.project.id=34f80a261b4141b79ce3bae0df60bf5b&role.id=7c12d43981d3489c96be96ccaf741270 => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:44.678472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:44.768477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0378f-8559-4b85-978e-7775f0e2580c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:44.791990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af40b8-0cf0-4b33-8a80-d94444a202d7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:44.843200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:44.843873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.844152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.844493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da/users/a5af0635781d459eb4cc38af780ab735/roles/7ef4aa4b4180492cb81a40024c482a71` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.849212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] RBAC: Authorizing `identity:create_grant(project_id=6067280cdc4440acac78a21b0daba4da, user_id=a5af0635781d459eb4cc38af780ab735, role_id=7ef4aa4b4180492cb81a40024c482a71)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:44.852146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:44.866053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a37a12-f9a0-4f0d-8778-7c6a9a3f38cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:44.866794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37a12-f9a0-4f0d-8778-7c6a9a3f38cd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.867119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37a12-f9a0-4f0d-8778-7c6a9a3f38cd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.867442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a37a12-f9a0-4f0d-8778-7c6a9a3f38cd None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.872621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37a12-f9a0-4f0d-8778-7c6a9a3f38cd None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:44.875905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 92/178] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:44 2022] PUT /identity/v3/projects/6067280cdc4440acac78a21b0daba4da/users/a5af0635781d459eb4cc38af780ab735/roles/7ef4aa4b4180492cb81a40024c482a71 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:44.877423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a37a12-f9a0-4f0d-8778-7c6a9a3f38cd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:44.879826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 87/179] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:44 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:44.886279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:44.968688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:44.987842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3475a9-eea4-4a82-9336-f2354b32db0f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:44.988170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3475a9-eea4-4a82-9336-f2354b32db0f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:44.988384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3475a9-eea4-4a82-9336-f2354b32db0f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:44.990372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 88/180] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:45.000013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.000324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.000609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.044668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:45.046710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:45.050938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde450d0-5205-4aac-9b13-6f6770c1a75e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:45.056325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:45.125092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:45.125727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.126003 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.126300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.127060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] RBAC: Adding query filter params (role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=e55c08d9c6404546b2c05b48636468bc) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:45.127368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:45.128868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:45.145056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 93/181] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:44 2022] GET /identity/v3/role_assignments?user.id=e55c08d9c6404546b2c05b48636468bc&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:45.176623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 89/182] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:45.477282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:45.591192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cd23e7-45de-4450-a93d-c79edf62ce34 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:45.607648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:45.642288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d9887-cfea-4f99-a235-1e4b444bd5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:45.642980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d9887-cfea-4f99-a235-1e4b444bd5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.643474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d9887-cfea-4f99-a235-1e4b444bd5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.643883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d9887-cfea-4f99-a235-1e4b444bd5c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.655716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d9887-cfea-4f99-a235-1e4b444bd5c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2f5d9887-cfea-4f99-a235-1e4b444bd5c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:45.661566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:45.664011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:45.665992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 94/183] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:45.671585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:45.675949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573e47-0b08-4bdb-b9eb-fff015f58050 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.676232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573e47-0b08-4bdb-b9eb-fff015f58050 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.677182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a573e47-0b08-4bdb-b9eb-fff015f58050 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.678894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 95/184] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:45.684231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c06742-7b0d-4b4a-930e-34eca2beda41 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.684515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c06742-7b0d-4b4a-930e-34eca2beda41 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.684790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c06742-7b0d-4b4a-930e-34eca2beda41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.704509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-56c06742-7b0d-4b4a-930e-34eca2beda41 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:45.705708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-56c06742-7b0d-4b4a-930e-34eca2beda41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:45.778410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 96/185] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:45.783530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d145-af75-489a-86bd-3a8396850a09 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.783889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d145-af75-489a-86bd-3a8396850a09 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.784238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5660d145-af75-489a-86bd-3a8396850a09 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.790176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 97/186] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:45.791535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.791921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.792234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.811392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:45.812774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:45.818764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e0e3a-31a4-43f1-a25b-7ded36ed3cbb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:45.855189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:45.855894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:45.856241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:45.856580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:45.857304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:45.857671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:45.859662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:45.871850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 90/187] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:45.878115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:45.883026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:45.892527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 98/188] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:45.990182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c2b036-5356-4723-a0bb-a167b9508eb5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.025143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.025777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.026109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.026407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] PATH_INFO: `/v3/users/1b3d98d4475b43f1a168aa8570701953` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.041666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] RBAC: Authorizing `identity:get_user(user_id=1b3d98d4475b43f1a168aa8570701953)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.043180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.044794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 91/189] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:45 2022] GET /identity/v3/users/1b3d98d4475b43f1a168aa8570701953 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:46.055680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.055820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.097108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.132362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.179926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6baa1-ee94-4d0d-8467-7adbe1e813ce None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.180802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0675f0-be65-449d-8f3b-98e05e9fab21 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.215440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.216074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.216354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.216615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.218722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.219334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.219610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.219878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.226943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:46.231749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:46.232581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:46.233351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 92/190] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:06:46.234627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.291987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.293360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.326947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.343917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391e9704-8211-4581-90e5-bcfe0ec59966 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.365842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.383128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.383754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.384024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.384301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.385403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.386885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.388434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 93/191] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:46.390473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.399670 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 99/192] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:46.400727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.499562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3da2e-c6a3-4bdb-a88e-b80681421f2e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.501850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7754690a-46a6-4e06-a5dc-3b1c1d8758b8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.533187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.533797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.534078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.534377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.535081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.537114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.537482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.538118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.538425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.538696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.539353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:46.539649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.540988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.545200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.550611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 94/193] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:46.551316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.570240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 100/194] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:46 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:46.570240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.606472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.661901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897c047b-02b2-4c93-9e56-f877ce4c424c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.688160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c145a-5123-4f24-ad11-b4eff1539446 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.698598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.699412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.699785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.700154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.701438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.703030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.704976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 95/195] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:46.711379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.729508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.730586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.731009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.731530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] PATH_INFO: `/v3/users/1b3d98d4475b43f1a168aa8570701953` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.733135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3d98d4475b43f1a168aa8570701953)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.735066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.737156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 101/196] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/users/1b3d98d4475b43f1a168aa8570701953 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:46.743345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:46.813993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.824058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92acf93e-f6f3-4295-a896-72caaa3b48d8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.843239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:46.872483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f35dde-8e9f-4f92-98f4-f3189ed430c9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:46.877295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.877921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.878215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.878510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:46.879245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=1b3d98d4475b43f1a168aa8570701953) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:46.879552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:46.881476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:46.889176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 96/197] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/role_assignments?user.id=1b3d98d4475b43f1a168aa8570701953&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:46.915296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:46.915928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:46.916203 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:46.916472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:47.003672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:47.005562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:47.080227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:47.127920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 102/198] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 10:06:47.133947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:47.247212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d03f3-a8d5-4e5a-864e-4c905be819bf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:47.288339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2056f1-6079-439e-b229-6c3599b86fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:47.288988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2056f1-6079-439e-b229-6c3599b86fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:47.289265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2056f1-6079-439e-b229-6c3599b86fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:47.289540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2056f1-6079-439e-b229-6c3599b86fe0 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:47.290689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2056f1-6079-439e-b229-6c3599b86fe0 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:47.292119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2056f1-6079-439e-b229-6c3599b86fe0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:47.293760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 97/199] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:47 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:47.299800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:47.439099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a165b663-8b65-4454-a428-803cf5ebaece None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:47.487505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d46b0-4959-4492-b437-eb50cb62af9e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:47.487828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d46b0-4959-4492-b437-eb50cb62af9e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:47.488055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d46b0-4959-4492-b437-eb50cb62af9e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:47.490293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 98/200] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:47.490977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:47.491604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:47.491981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:47.492217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:47.492975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] RBAC: Adding query filter params (role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=1b3d98d4475b43f1a168aa8570701953) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:47.493289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:47.494799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:47.497150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:47.497423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:47.497700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:47.505317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 103/201] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:47 2022] GET /identity/v3/role_assignments?user.id=1b3d98d4475b43f1a168aa8570701953&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:47.519812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:47.521732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:47.604032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:47.636190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 99/202] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:48.044370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:48.071251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:48.164594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6439a3-b710-4927-8115-c7fc41edd154 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:48.202605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:48.203226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:48.203503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:48.203774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:48.222324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:48.228455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:48.229302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:48.230077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 104/203] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:48.235497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:48.390899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a24a691-5b48-4252-bf3c-78d7444483d2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:48.443132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:48.443756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:48.444038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:48.444315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:48.444978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:48.445279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:48.446686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:48.457502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 100/204] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:48.467295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:48.588623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:48.608189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64ef86-f6b0-45d7-a4b7-253216e739d2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:48.670650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:48.671202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:48.671478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:48.671747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] PATH_INFO: `/v3/users/a5af0635781d459eb4cc38af780ab735` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:48.679404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] RBAC: Authorizing `identity:get_user(user_id=a5af0635781d459eb4cc38af780ab735)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:48.682079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:48.683831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 105/205] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:48 2022] GET /identity/v3/users/a5af0635781d459eb4cc38af780ab735 => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:48.693531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:48.753785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:48.825977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d7d8a-49b8-429e-a601-44735c8824fd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:48.873300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:48.873918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:48.874193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:48.874485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:48.965939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:48.967345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:49.022759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:49.095869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 101/206] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:49.101890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:49.226890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41174a1-aa5e-4a43-8e2b-f4d5e38d5fd5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:49.267058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea00af9-e016-4eeb-91fc-e7c4697fb7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:49.267690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea00af9-e016-4eeb-91fc-e7c4697fb7af None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.267967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea00af9-e016-4eeb-91fc-e7c4697fb7af None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.268237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea00af9-e016-4eeb-91fc-e7c4697fb7af None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.269357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea00af9-e016-4eeb-91fc-e7c4697fb7af None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:49.271406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea00af9-e016-4eeb-91fc-e7c4697fb7af None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:49.273084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 106/207] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:49 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:49.278932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:49.402909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7054ec81-a100-46d1-a330-a2cbf3d90bf7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:49.460050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:49.460683 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.460961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.461231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.461967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] RBAC: Adding query filter params (role.id=7ef4aa4b4180492cb81a40024c482a71, scope.project.id=6067280cdc4440acac78a21b0daba4da, user.id=a5af0635781d459eb4cc38af780ab735) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:49.462294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:49.463734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736621f-79c9-4e7d-9101-bb4a1583791a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:49.475928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 102/208] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:49 2022] GET /identity/v3/role_assignments?user.id=a5af0635781d459eb4cc38af780ab735&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=7ef4aa4b4180492cb81a40024c482a71 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:49.580319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306206-a801-4c96-91cc-5b44c7c6c573 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.580606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306206-a801-4c96-91cc-5b44c7c6c573 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.580794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37306206-a801-4c96-91cc-5b44c7c6c573 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.582319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 107/209] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:49.588490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44898810-2b85-41a2-a43d-8dbda88bedaf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.588776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44898810-2b85-41a2-a43d-8dbda88bedaf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.589032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44898810-2b85-41a2-a43d-8dbda88bedaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.609579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:49.622579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-44898810-2b85-41a2-a43d-8dbda88bedaf None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:49.623785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-44898810-2b85-41a2-a43d-8dbda88bedaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:49.729872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 103/210] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:49.794046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a73b5a-3dc1-400f-bf5e-cc0af28245e9 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.794611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a73b5a-3dc1-400f-bf5e-cc0af28245e9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.794611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a73b5a-3dc1-400f-bf5e-cc0af28245e9 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.795968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 108/211] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:49.801442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.801977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.802470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.827171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:49.828349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:49.856728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b4ebc-be43-4f01-9d40-2ce6248dab4a None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.857316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b4ebc-be43-4f01-9d40-2ce6248dab4a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.857812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b4ebc-be43-4f01-9d40-2ce6248dab4a None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.859414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 109/212] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:49.864916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:49.865447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:49.865916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:49.886439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:49.887625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:49.902667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:49.939049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 104/213] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:49.969491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 110/214] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:06:50.027050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.095214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.131329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c650e-2a52-4a29-a734-07b61fe93036 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.167027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.167663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.167936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.168209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.179378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:50.184270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:50.185105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:50.185884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 105/215] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:50.191374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.236041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.313635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4bf1a4-f402-411c-83d8-13f044ad1f82 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.348991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c322cd1-895c-46b5-8f43-9b9e987adec4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.350174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.350809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.351082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.351348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.352007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:50.352304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:50.353668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:50.361900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 111/216] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:50.362722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.375883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.377887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.386133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cf2b7-7113-430a-aadd-0b2a478fe066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.386730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cf2b7-7113-430a-aadd-0b2a478fe066 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.387004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cf2b7-7113-430a-aadd-0b2a478fe066 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.387271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cf2b7-7113-430a-aadd-0b2a478fe066 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.388360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cf2b7-7113-430a-aadd-0b2a478fe066 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:50.389792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cf2b7-7113-430a-aadd-0b2a478fe066 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:50.391368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 106/217] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:50.392460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc144428-aba0-43b0-9724-170479200f79 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.392728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc144428-aba0-43b0-9724-170479200f79 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.392992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc144428-aba0-43b0-9724-170479200f79 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.394167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 107/218] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:50.399095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc144428-aba0-43b0-9724-170479200f79 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.472561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b593af-989d-43cc-91f1-dd9357dbadda None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.501798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc144428-aba0-43b0-9724-170479200f79 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.506644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.507272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.507550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.507822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.517585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:50.523620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:50.524698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 112/219] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:50.525672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.539498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea28241-1b67-4862-9256-ec821f551a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.540129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea28241-1b67-4862-9256-ec821f551a96 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.540401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea28241-1b67-4862-9256-ec821f551a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.540667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea28241-1b67-4862-9256-ec821f551a96 None admin] PATH_INFO: `/v3/users/27f0e2b9a8a242a3a66ff8a3400a4f5b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.553883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea28241-1b67-4862-9256-ec821f551a96 None admin] RBAC: Authorizing `identity:get_user(user_id=27f0e2b9a8a242a3a66ff8a3400a4f5b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:50.555390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea28241-1b67-4862-9256-ec821f551a96 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:50.557004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 108/220] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/users/27f0e2b9a8a242a3a66ff8a3400a4f5b => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:50.562799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.563079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.563352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.581035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:50.582215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:50.594600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.628880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018a0fa6-2ca6-4561-85db-e0a25649971d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.631711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.653232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 109/221] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:50.654001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.662863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.663470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.663747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.664016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.664660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:50.666103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:50.689319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 113/222] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:50 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:50.691309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.798641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63910f63-d1ca-4d27-847c-950655a5e83a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.841119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.842117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.842658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.843169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.844053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:50.844560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:50.846163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:50.856827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 110/223] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:50.862723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:50.868705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ed0285-b6ee-4b0f-ad3b-3e9e243ff3f1 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.897567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.915987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:50.916531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:50.916810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:50.917082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:50.920224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:50.987064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cd8e1-2ca2-4062-83af-aa4fe7c4e2c7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:50.998311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:50.999930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.030056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.030289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.030459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.030733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] PATH_INFO: `/v3/users/1b3d98d4475b43f1a168aa8570701953` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.031924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] RBAC: Authorizing `identity:get_user(user_id=1b3d98d4475b43f1a168aa8570701953)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.033373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.035388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 111/224] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/users/1b3d98d4475b43f1a168aa8570701953 => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:51.036300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.117701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 114/225] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:51.118938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.146629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:51.164109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04693ef5-0210-4df2-b255-bb83a99033ce None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.183747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:51.210129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.211140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.212912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.213783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.227619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:51.234011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:51.234917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:51.235696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:51.242525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 112/226] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:51.243760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.249639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8ff4fd-d010-4a3c-b2ac-a1c45e546dea None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.293856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.294519 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.294794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.295587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.296739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.298171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.300293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 115/227] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:51.302807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.368046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82204b27-71ef-4544-b780-11db8e8f00f6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.403929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:51.408270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.409001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.409475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.409813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.411091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.412623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.414036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072922bb-cfdf-46e7-a08e-acebdb82e7f8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.414401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 113/228] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:51.415215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.466061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:51.466474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.468331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.469137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.470187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.470813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:51.471288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.474752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.490158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 116/229] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:51.490158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.534238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bc3c89-d49a-4944-8fad-63bb7fe60d1a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.570783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.571449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.571772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.572050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/1b3d98d4475b43f1a168aa8570701953/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.574015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e8d106e8b48f3be72ac35d6185817, user_id=1b3d98d4475b43f1a168aa8570701953, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.575504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.591245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 114/230] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:51 2022] PUT /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/1b3d98d4475b43f1a168aa8570701953/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:51.599381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.616873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0d66a-cb63-4a37-b051-58e0e0517c51 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.702818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.703777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.704196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.704623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.705516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=27f0e2b9a8a242a3a66ff8a3400a4f5b) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:51.705968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.708552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.721166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 117/231] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/role_assignments?user.id=27f0e2b9a8a242a3a66ff8a3400a4f5b&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=7c12d43981d3489c96be96ccaf741270 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:51.794166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:51.797967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffdcacf-2348-42ac-9ba1-d4d8874c65bf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:51.843006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:51.843927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:51.844363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:51.844789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] PATH_INFO: `/v3/users/1b3d98d4475b43f1a168aa8570701953` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:51.846079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] RBAC: Authorizing `identity:get_user(user_id=1b3d98d4475b43f1a168aa8570701953)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:51.847753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:51.850163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 115/232] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/users/1b3d98d4475b43f1a168aa8570701953 => generated 283 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:51.859396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:51.888102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:51.984910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eae6eb3-d193-426f-95be-b29a9304bf74 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:52.022598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3c3503-24f9-4cec-bbaa-9c0e5463ab53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:52.023227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3503-24f9-4cec-bbaa-9c0e5463ab53 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:52.023499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3503-24f9-4cec-bbaa-9c0e5463ab53 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:52.023818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3c3503-24f9-4cec-bbaa-9c0e5463ab53 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:52.024872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c3503-24f9-4cec-bbaa-9c0e5463ab53 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:52.026351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3c3503-24f9-4cec-bbaa-9c0e5463ab53 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:52.028145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 118/233] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:51 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:52.033639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:52.139072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f19f027-e8eb-4cde-a777-05963614bb3f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:52.174710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71752755-3520-4f08-b2d0-20eb4a3902ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:52.175382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752755-3520-4f08-b2d0-20eb4a3902ac None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:52.175662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752755-3520-4f08-b2d0-20eb4a3902ac None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:52.175940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71752755-3520-4f08-b2d0-20eb4a3902ac None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/1b3d98d4475b43f1a168aa8570701953/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:52.178947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71752755-3520-4f08-b2d0-20eb4a3902ac None admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e8d106e8b48f3be72ac35d6185817, user_id=1b3d98d4475b43f1a168aa8570701953, role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:52.180388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71752755-3520-4f08-b2d0-20eb4a3902ac None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:52.199685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 116/234] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:52 2022] PUT /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/1b3d98d4475b43f1a168aa8570701953/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:53.432863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28dae-c50c-4812-94dd-fc49867c0130 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:53.433402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28dae-c50c-4812-94dd-fc49867c0130 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:53.433402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a28dae-c50c-4812-94dd-fc49867c0130 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:53.434991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0be0e1-a908-4870-a0c5-6fe2a4fbc41f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:53.435382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 119/235] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:53.435788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0be0e1-a908-4870-a0c5-6fe2a4fbc41f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:53.436212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0be0e1-a908-4870-a0c5-6fe2a4fbc41f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:53.437659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 117/236] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:06:53.439725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:53.440002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:53.440273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:53.444201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:53.444698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:53.445159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:53.462496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:53.463681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:53.464721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:53.466254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:53.512342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:53.538229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 120/237] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:53.594458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 118/238] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:53.815675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:53.835079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:53.904436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:53.919874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e495f-3ab4-420a-9d64-d5b2c883e7b0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:53.956543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:53.957179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:53.957972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:53.958326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:53.968867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:53.973806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:53.974760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:53.975908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 121/239] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:53 2022] GET /identity/v3/roles/reader => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:53.980723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:54.063487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:54.086680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bfb99d-a1dc-4f79-94c8-b404b2236502 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:54.088486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77faa85d-048e-4cca-a847-52585dd780b2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:54.125681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:54.126324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.129926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.131453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.132151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:54.132470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:54.137049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:54.156948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:54.157644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.157924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.158449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.161743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 122/240] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:54.166882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:54.168454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:54.170329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 119/241] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:54.172179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:54.178730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:54.223784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:54.329918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e1290-ecd9-4497-ac4a-ebf5ad84800b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:54.343610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f089dd-59cf-4645-8dfc-d6aca3cc4c9a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:54.372400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bed4b4-d2fb-4405-b061-94d86d24861d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:54.373008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bed4b4-d2fb-4405-b061-94d86d24861d None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.373281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bed4b4-d2fb-4405-b061-94d86d24861d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.373551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bed4b4-d2fb-4405-b061-94d86d24861d None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.374197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bed4b4-d2fb-4405-b061-94d86d24861d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:54.375640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bed4b4-d2fb-4405-b061-94d86d24861d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:54.390737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:54.391463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.391795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.392093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] PATH_INFO: `/v3/users/ee8a9fae99534e848c9f9db4dd776183` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.406292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 120/242] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:54 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:54.407881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd446227-66d0-4503-ab04-9b8d14c56fd8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.408172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd446227-66d0-4503-ab04-9b8d14c56fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.408448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd446227-66d0-4503-ab04-9b8d14c56fd8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.410010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 121/243] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:54.420410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aa80e-0bd4-467e-a646-834722d40c84 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.420677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aa80e-0bd4-467e-a646-834722d40c84 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.420952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957aa80e-0bd4-467e-a646-834722d40c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.422616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:54.430915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] RBAC: Authorizing `identity:get_user(user_id=ee8a9fae99534e848c9f9db4dd776183)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:54.432451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:54.434603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 123/244] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:54 2022] GET /identity/v3/users/ee8a9fae99534e848c9f9db4dd776183 => generated 297 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:54.443266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:54.468263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-957aa80e-0bd4-467e-a646-834722d40c84 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:54.469504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-957aa80e-0bd4-467e-a646-834722d40c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:54.577746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 122/245] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:54.635087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68380c9a-bde7-4f34-9d08-45d5171377bd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:54.690748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a258c97c-2e59-4311-8604-555f1fb69ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:54.691395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258c97c-2e59-4311-8604-555f1fb69ec5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.691674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258c97c-2e59-4311-8604-555f1fb69ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.692544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a258c97c-2e59-4311-8604-555f1fb69ec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.784310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:54.789075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258c97c-2e59-4311-8604-555f1fb69ec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:54.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a258c97c-2e59-4311-8604-555f1fb69ec5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:54.819932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e5cd2-4e7a-462b-8b69-71f234631edd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.820213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e5cd2-4e7a-462b-8b69-71f234631edd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.820609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826e5cd2-4e7a-462b-8b69-71f234631edd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.822177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 123/246] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:54.827137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afba17-fff7-4757-ad41-97bdc56a9073 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.827412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afba17-fff7-4757-ad41-97bdc56a9073 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.827685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59afba17-fff7-4757-ad41-97bdc56a9073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.852555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-59afba17-fff7-4757-ad41-97bdc56a9073 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:54.853617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-59afba17-fff7-4757-ad41-97bdc56a9073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:54.865593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:54.916001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 124/247] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:54.917209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0826-74f3-4c5f-82b1-99b50674ab4d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.917494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0826-74f3-4c5f-82b1-99b50674ab4d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.917768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb0826-74f3-4c5f-82b1-99b50674ab4d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.919380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 125/248] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:54.923192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0826-74f3-4c5f-82b1-99b50674ab4d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:54.942357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 124/249] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:54.943986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:54.944328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:54.944666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:54.968169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:54.969366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:55.053749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 125/250] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:55.055162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.055313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:55.065348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb0826-74f3-4c5f-82b1-99b50674ab4d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.115284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.116737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.117367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.117917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.119550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:55.121677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:55.124506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 126/251] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:54 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:55.130989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.157140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:55.182076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec05cce9-893e-4616-9a01-791ac0e79b31 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.245059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.246981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.247500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.247966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.259795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:55.265356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:55.266470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:06:55.267254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:55.270859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 126/252] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:55.278727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.335934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d95cc-d5ce-4c18-ab5b-13e8c2ab2b70 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.399673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.400582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.401072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.401514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.402600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] RBAC: Adding query filter params (role.id=7ef4aa4b4180492cb81a40024c482a71, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=ee8a9fae99534e848c9f9db4dd776183) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:55.404345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:55.405966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:55.415690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 127/253] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/role_assignments?user.id=ee8a9fae99534e848c9f9db4dd776183&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=7ef4aa4b4180492cb81a40024c482a71 => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:55.419700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.440464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758b0ca0-afa5-4bc3-a1d4-9b06701b0149 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.464799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:55.490607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:55.493036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.493978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.494423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.494836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.495650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:55.496094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:55.497658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:55.516199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 127/254] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:55.518890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.573420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c925d82b-5bd1-439a-9e6f-bfcd585abc63 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.622182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.623115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.623817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.624235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.640206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:55.646975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:55.648073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:55.649068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:55.651944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 128/255] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:55.653142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.712457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203aa2ee-60b7-40df-8157-ff504b9a225d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.774077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.777022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.777297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.777562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.799576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:55.807050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:55.808131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:06:55.809146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 128/256] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:55.809974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.831699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:55.845612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0e0741-0286-444e-8d7f-bd17de93b8b5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:55.908450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:55.909259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:55.909902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:55.910176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:55.910550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] PATH_INFO: `/v3/users/1b3d98d4475b43f1a168aa8570701953` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:55.912203 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] RBAC: Authorizing `identity:get_user(user_id=1b3d98d4475b43f1a168aa8570701953)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:55.913711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:55.915545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 129/257] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/users/1b3d98d4475b43f1a168aa8570701953 => generated 283 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:55.916083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:55.999859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b10c94-1269-4bd0-94e9-0bbe6be9aebc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.074768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a7a9f-2c67-4313-be54-998595b2347d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.084045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.084965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.085442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.085909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.086888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:56.087391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.089004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.109564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 129/258] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:56.110928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:56.120458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.120458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.120458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.120458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.120789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:56.120789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.120940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.129594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 130/259] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:06:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:56.130450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:56.166558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:56.294527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c4852-1ae2-4446-8065-e99a8bb94fca None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.311612 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:56.320167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153bfa7-22ba-4ad3-bf3d-d7bb0267f50a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.333950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.334861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.335309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.335779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.393655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.394324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.394601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.395027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] PATH_INFO: `/v3/users/27f0e2b9a8a242a3a66ff8a3400a4f5b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.397120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] RBAC: Authorizing `identity:get_user(user_id=27f0e2b9a8a242a3a66ff8a3400a4f5b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.400117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.401893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 131/260] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:56 2022] GET /identity/v3/users/27f0e2b9a8a242a3a66ff8a3400a4f5b => generated 297 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:56.402510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:56.439104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.440575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.577868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 130/261] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:56.578690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:56.608076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:56.637335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:56.684322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b426190-cac1-4e71-b0c1-f99a10527c0b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.696097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1226ec58-1c91-48ac-88c8-08b798a5cc0e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.719871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.720663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.720942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.721217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.722375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.723870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.725704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 131/262] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:56 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:56.726264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:56.794471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.795106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.795379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.795645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] PATH_INFO: `/v3/users/1b3d98d4475b43f1a168aa8570701953` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.799718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] RBAC: Authorizing `identity:get_user(user_id=1b3d98d4475b43f1a168aa8570701953)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.801193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.804408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 132/263] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:56 2022] GET /identity/v3/users/1b3d98d4475b43f1a168aa8570701953 => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:56.805449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:56.838631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:56.850879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8a407b-870f-4c72-bcc6-67415703a4a7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:56.887939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:56.888571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:56.888846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:56.889166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:56.890966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:56.892819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:56.894494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 132/264] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:56 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:56.895838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:57.001825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca35428e-93a2-4231-b5cb-b06b56b3fbae None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:57.053809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:57.054452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.054726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.054996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.082615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:57.084290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e1f72-8477-41ff-a32c-aa0e30e69bd4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:57.128555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:57.130093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:57.157392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:57.158067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.158412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.158684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/27f0e2b9a8a242a3a66ff8a3400a4f5b/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.165247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e8d106e8b48f3be72ac35d6185817, user_id=27f0e2b9a8a242a3a66ff8a3400a4f5b, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:57.166753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:57.190512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 133/265] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:56 2022] PUT /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/27f0e2b9a8a242a3a66ff8a3400a4f5b/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:57.191430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:57.236411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:57.295331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 133/266] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:06:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:06:57.306556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:57.430678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:57.452482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca901e2d-2c3b-483d-95ae-a62cd0b1cca5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:57.466979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce4aad3-97af-4be9-becf-da78e0f80c8f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:57.522247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd853c20-2ea0-437e-9010-76902f6017ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:57.523100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853c20-2ea0-437e-9010-76902f6017ba None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.524246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853c20-2ea0-437e-9010-76902f6017ba None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.524246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd853c20-2ea0-437e-9010-76902f6017ba None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.524960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd853c20-2ea0-437e-9010-76902f6017ba None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:57.526913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd853c20-2ea0-437e-9010-76902f6017ba None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:57.527518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:57.528370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.528845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.529905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 134/267] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:57 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:57.530304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372319-9488-40e6-a2c2-b82c8ca0f80b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.530588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372319-9488-40e6-a2c2-b82c8ca0f80b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.530820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.530916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0372319-9488-40e6-a2c2-b82c8ca0f80b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.531986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=1b3d98d4475b43f1a168aa8570701953) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:57.534267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 135/268] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:57.534661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:57.535197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372319-9488-40e6-a2c2-b82c8ca0f80b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:57.536924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b478d6-b836-4283-81d6-0126111efd62 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:57.549405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 134/269] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:57 2022] GET /identity/v3/role_assignments?user.id=1b3d98d4475b43f1a168aa8570701953&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:57.552093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89238de-b264-42fa-8728-628954e5fc9d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.552702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89238de-b264-42fa-8728-628954e5fc9d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.553213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89238de-b264-42fa-8728-628954e5fc9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.585957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c89238de-b264-42fa-8728-628954e5fc9d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:57.587230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c89238de-b264-42fa-8728-628954e5fc9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:57.718435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0372319-9488-40e6-a2c2-b82c8ca0f80b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:57.748166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:57.768568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:57.769431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.770445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.771610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.772475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] RBAC: Adding query filter params (role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=1b3d98d4475b43f1a168aa8570701953) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:57.772891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:57.774512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df254db3-26cb-4073-9b93-c49d36ab3d38 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:57.780148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 135/270] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:57.781628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8276-08fc-4799-9fce-a8fb81fe373d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.781920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8276-08fc-4799-9fce-a8fb81fe373d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.782072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 136/271] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:06:57 2022] GET /identity/v3/role_assignments?user.id=1b3d98d4475b43f1a168aa8570701953&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:57.783304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313f8276-08fc-4799-9fce-a8fb81fe373d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.784590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 136/272] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:57.792372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:57.792816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:57.793194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:57.831379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:57.833898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:57.938474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 137/273] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:58.181544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8276-08fc-4799-9fce-a8fb81fe373d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.195733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:58.243678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.304360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313f8276-08fc-4799-9fce-a8fb81fe373d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:58.344293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13eaf77-1fd0-47cc-8380-a76df421df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:58.345043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaf77-1fd0-47cc-8380-a76df421df56 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.345318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaf77-1fd0-47cc-8380-a76df421df56 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.345588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13eaf77-1fd0-47cc-8380-a76df421df56 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.360815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13eaf77-1fd0-47cc-8380-a76df421df56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a13eaf77-1fd0-47cc-8380-a76df421df56 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:58.364221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:58.365573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:58.366437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 137/274] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:06:58.367072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4a77-0820-4e2d-bb7b-384ad6f4e225 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.367072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4a77-0820-4e2d-bb7b-384ad6f4e225 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.367072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371c4a77-0820-4e2d-bb7b-384ad6f4e225 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.369185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 138/275] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:58.375307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4a77-0820-4e2d-bb7b-384ad6f4e225 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.393378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a8ff39-5a99-4eeb-aa21-38efd7ab041c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:58.407564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:58.427994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b009a56d-2820-40e6-9315-9f8c102f7ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:58.428734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009a56d-2820-40e6-9315-9f8c102f7ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.429008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009a56d-2820-40e6-9315-9f8c102f7ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.429275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b009a56d-2820-40e6-9315-9f8c102f7ab4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.430442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009a56d-2820-40e6-9315-9f8c102f7ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:58.431920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b009a56d-2820-40e6-9315-9f8c102f7ab4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:58.433418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 138/276] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:06:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:58.434533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.434802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.435066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.453763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:58.455002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:58.461572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:58.497094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371c4a77-0820-4e2d-bb7b-384ad6f4e225 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:58.524975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 139/277] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:06:58.527038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.542871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:58.543847 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.544317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.544780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.545661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:58.546302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:58.547962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:58.558057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 139/278] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:06:58.568779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.629941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fad7dd-fdd9-45b0-8c64-7b9f7fa91a9e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:58.647997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:58.671279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:58.671888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.672164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.672431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.673070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:58.674536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:58.705750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 140/279] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:06:58 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:06:58.710234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbbb79-91f5-4971-a0bf-2e8dac6958c7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:58.745258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:58.750739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:58.751378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.751724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.752004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] PATH_INFO: `/v3/users/308338bf4a8b4f7b907c62cc19910283` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.768076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] RBAC: Authorizing `identity:get_user(user_id=308338bf4a8b4f7b907c62cc19910283)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:58.769779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:58.772594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 140/280] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:58 2022] GET /identity/v3/users/308338bf4a8b4f7b907c62cc19910283 => generated 293 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:58.779384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.838936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:58.904592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a76840d-f33a-4c3d-993e-a6124cadc340 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:58.938837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:58.939785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:58.940362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:58.940822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:58.941851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, user.id=308338bf4a8b4f7b907c62cc19910283, scope.system=all) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:58.942158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:58.943629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:58.950252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 141/281] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Oct 4 10:06:58 2022] GET /identity/v3/role_assignments?user.id=308338bf4a8b4f7b907c62cc19910283&scope.system=all&role.id=7c12d43981d3489c96be96ccaf741270 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:58.979643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e5f63-554d-421a-aa55-fbb08f4340a5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:59.022523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:59.023454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.023882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.024298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.037737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:06:59.043022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:06:59.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:06:59.045078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:06:59.047191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 141/282] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:06:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:06:59.053294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:59.091642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:59.172450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0128ba6-ddb7-4471-b32a-79c18725291d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:59.209915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:59.210562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.210839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.211109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.211764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:06:59.212064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:59.213430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:59.222352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 142/283] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:06:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:59.228181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:59.283285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:59.343758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54eef60-645c-4d4e-83e9-e2f1ab5e8d2f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:59.393433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:59.394389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.394899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.395346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] PATH_INFO: `/v3/users/ee8a9fae99534e848c9f9db4dd776183` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.398999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] RBAC: Authorizing `identity:get_user(user_id=ee8a9fae99534e848c9f9db4dd776183)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:59.400663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:59.402857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 142/284] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:06:59 2022] GET /identity/v3/users/ee8a9fae99534e848c9f9db4dd776183 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:59.408098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:59.515624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:59.523406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619386fb-d2ef-439d-95d6-1004bb3bdc5f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:59.559322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982142f-17e3-4469-ace4-7b827f081e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:59.559881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982142f-17e3-4469-ace4-7b827f081e6e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.560158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982142f-17e3-4469-ace4-7b827f081e6e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.560431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982142f-17e3-4469-ace4-7b827f081e6e None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.561550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982142f-17e3-4469-ace4-7b827f081e6e None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:59.563024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982142f-17e3-4469-ace4-7b827f081e6e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:59.564788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 143/285] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:06:59 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:06:59.571124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:06:59.694365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751bacdd-3b67-4972-ae6e-025689da595c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:06:59.714680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:06:59.734043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2aaef1-605c-466b-9233-1be31ca21922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:06:59.734883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aaef1-605c-466b-9233-1be31ca21922 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.735167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aaef1-605c-466b-9233-1be31ca21922 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.735493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2aaef1-605c-466b-9233-1be31ca21922 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/ee8a9fae99534e848c9f9db4dd776183/roles/7ef4aa4b4180492cb81a40024c482a71` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.737546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2aaef1-605c-466b-9233-1be31ca21922 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e8d106e8b48f3be72ac35d6185817, user_id=ee8a9fae99534e848c9f9db4dd776183, role_id=7ef4aa4b4180492cb81a40024c482a71)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:06:59.739605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2aaef1-605c-466b-9233-1be31ca21922 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:06:59.765347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 143/286] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:06:59 2022] PUT /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/users/ee8a9fae99534e848c9f9db4dd776183/roles/7ef4aa4b4180492cb81a40024c482a71 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:06:59.843445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8f129-3c51-4d41-92c9-4b8566a61f4c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.843926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8f129-3c51-4d41-92c9-4b8566a61f4c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.843926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8f129-3c51-4d41-92c9-4b8566a61f4c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.845648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 144/287] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:06:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:06:59.851478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:06:59.851927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:06:59.852107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:06:59.880173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:06:59.881536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:06:59.980449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 144/288] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:06:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:00.361844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8f129-3c51-4d41-92c9-4b8566a61f4c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:00.495393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8f129-3c51-4d41-92c9-4b8566a61f4c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:00.529294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:00.529927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:00.530213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:00.530487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:00.541648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:00.546540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:00.547379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:00.548225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 145/289] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:00.554284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:00.677487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1b85f4-9e71-40a3-aa01-d7112ffba26d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:00.690656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:00.719017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:00.719638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:00.719913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:00.720178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:00.720855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:00.721154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:00.722681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:00.736084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 145/290] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:00.742324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:00.772964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:00.861647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd742aab-161d-4fb7-98fc-f11a68628c55 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:00.897216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:00.897771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:00.898047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:00.898354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] PATH_INFO: `/v3/users/27f0e2b9a8a242a3a66ff8a3400a4f5b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:00.899477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] RBAC: Authorizing `identity:get_user(user_id=27f0e2b9a8a242a3a66ff8a3400a4f5b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:00.900928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:00.902526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 146/291] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:00 2022] GET /identity/v3/users/27f0e2b9a8a242a3a66ff8a3400a4f5b => generated 297 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:00.908311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:01.022561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7408df32-854b-4b79-830f-4ba3db1271d8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:01.059279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:01.059895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.060175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.060447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.101232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:01.130807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:01.132272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:01.243613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 146/292] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:01.249142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:01.347001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:01.356740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703ab07-be41-4c46-bfa0-5a6db6a2ba0a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:01.391696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da2565a-f1b5-4d54-907f-1f5dbd268e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:01.392375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2565a-f1b5-4d54-907f-1f5dbd268e34 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.392654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2565a-f1b5-4d54-907f-1f5dbd268e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.392949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da2565a-f1b5-4d54-907f-1f5dbd268e34 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.394043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2565a-f1b5-4d54-907f-1f5dbd268e34 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:01.396031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da2565a-f1b5-4d54-907f-1f5dbd268e34 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:01.397647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 147/293] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:01 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:01.403740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:01.482188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:01.514565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481bb8e-99dd-46af-8271-e0a98a0bae5e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:01.548729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:01.549334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.549611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.549880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.550637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=27f0e2b9a8a242a3a66ff8a3400a4f5b) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:01.550998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:01.552509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55952782-c0d1-4f8f-916c-f16ca084bdb2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:01.558440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 147/294] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:01 2022] GET /identity/v3/role_assignments?user.id=27f0e2b9a8a242a3a66ff8a3400a4f5b&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=7c12d43981d3489c96be96ccaf741270 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:01.780022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e91c50-3a2d-431c-8473-f9ac4999b1c9 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.780346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e91c50-3a2d-431c-8473-f9ac4999b1c9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.780509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e91c50-3a2d-431c-8473-f9ac4999b1c9 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.782057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 148/295] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:01.788459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814dcd0-923d-434d-a69e-75e6b7019da7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.788802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814dcd0-923d-434d-a69e-75e6b7019da7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.789076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814dcd0-923d-434d-a69e-75e6b7019da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.795145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae8ab3-291b-43ef-b153-59e16f0cc004 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.795915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae8ab3-291b-43ef-b153-59e16f0cc004 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.796397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae8ab3-291b-43ef-b153-59e16f0cc004 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.797947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 149/296] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:01.805377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f2f9-477a-46f1-b1f5-84f75d99e3b9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.805745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f2f9-477a-46f1-b1f5-84f75d99e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.806190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed5f2f9-477a-46f1-b1f5-84f75d99e3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.813647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2814dcd0-923d-434d-a69e-75e6b7019da7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:01.815082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2814dcd0-923d-434d-a69e-75e6b7019da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:01.824498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6ed5f2f9-477a-46f1-b1f5-84f75d99e3b9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:01.825942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6ed5f2f9-477a-46f1-b1f5-84f75d99e3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:01.894241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 148/297] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:01.903033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073031a-0204-424d-9b47-550e0d912525 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.903766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073031a-0204-424d-9b47-550e0d912525 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.904373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073031a-0204-424d-9b47-550e0d912525 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.905505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 150/298] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:01.910227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 149/299] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:01.912295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:01.912607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:01.912869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:01.937646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:01.938960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:02.007818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 151/300] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:02.185223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073031a-0204-424d-9b47-550e0d912525 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.201160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.222065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:02.250619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:02.291612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073031a-0204-424d-9b47-550e0d912525 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.311407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab91ed62-4a15-470b-a705-83a38bfc8d5a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.327170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.327799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.328067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.328334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.329520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:02.331403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:02.333375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 150/301] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:02.338561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.352946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.353586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.353867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.354139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.365155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:02.369987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:02.370996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:02.371755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:02.375908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 152/302] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:02.376605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.444911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3becee58-880a-49d7-8316-9e0e974a0c9f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.457735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:02.479599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:02.481606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87985f7-1b69-4222-a411-a11f17419032 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.486257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.486895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.487172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.487443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.499769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:02.505739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:02.507090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 151/303] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:02.507898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.522261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.522956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.523293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.523721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] PATH_INFO: `/v3/groups` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.524719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:02.526142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:02.543410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 153/304] 10.222.0.13 () {60 vars in 1134 bytes} [Tue Oct 4 10:07:02 2022] POST /identity/v3/groups => generated 229 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:02.544295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.636463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a017400f-09a8-45dc-b56b-3256cd706341 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.675459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.676692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.676968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.677235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.678028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:02.678380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:02.679810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:02.690957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 152/305] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:02.695863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b52909a-bd3d-4d37-a181-13c17d02395b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.696645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.749856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.751293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.752036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.752496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.753680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:02.754350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:02.756830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:02.764505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:02.766131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:02.776272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 154/306] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:02.782450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.822758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080cfbf-440d-43c8-aa01-353a5e9d15fc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.862324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6e8b1e-775e-4276-b709-750d6c6cdbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.862957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8b1e-775e-4276-b709-750d6c6cdbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.863366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8b1e-775e-4276-b709-750d6c6cdbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.863707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6e8b1e-775e-4276-b709-750d6c6cdbd0 None admin] PATH_INFO: `/v3/users/308338bf4a8b4f7b907c62cc19910283` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.864841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e8b1e-775e-4276-b709-750d6c6cdbd0 None admin] RBAC: Authorizing `identity:get_user(user_id=308338bf4a8b4f7b907c62cc19910283)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:02.866325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6e8b1e-775e-4276-b709-750d6c6cdbd0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:02.867929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 153/307] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/users/308338bf4a8b4f7b907c62cc19910283 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:02.869052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3baf0-99a5-4ef8-842e-e373f4e0a914 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.869331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3baf0-99a5-4ef8-842e-e373f4e0a914 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.869603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf3baf0-99a5-4ef8-842e-e373f4e0a914 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.870965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 154/308] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:02.876503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.876781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.877051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.895674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:02.896836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:02.905165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf735b97-8211-45c8-9cb1-d042b73a71da None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:02.946698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:02.947308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:02.947517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:02.947789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] PATH_INFO: `/v3/users/cb4fadcd662c450795d835e82f59088c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:02.963274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] RBAC: Authorizing `identity:get_user(user_id=cb4fadcd662c450795d835e82f59088c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:02.965458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:02.972197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 155/309] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/users/cb4fadcd662c450795d835e82f59088c => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:02.972197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:02.978346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 155/310] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:02.980137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:03.050077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:03.066242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:03.075223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a486ab-6122-4e65-a20b-d411969f85b4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:03.112562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cfde32-5e20-47cf-bc45-d97a153d3649 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:03.117635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:03.118536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:03.118952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:03.119364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] PATH_INFO: `/v3/system/users/308338bf4a8b4f7b907c62cc19910283/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:03.121422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=308338bf4a8b4f7b907c62cc19910283, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:03.123456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:03.134653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 156/311] 10.222.0.13 () {58 vars in 1334 bytes} [Tue Oct 4 10:07:02 2022] PUT /identity/v3/system/users/308338bf4a8b4f7b907c62cc19910283/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:03.161487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:03.164395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:03.164974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:03.165407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:03.166444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] RBAC: Adding query filter params (role.id=7ef4aa4b4180492cb81a40024c482a71, user.id=cb4fadcd662c450795d835e82f59088c, scope.system=all) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:03.166959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:03.168596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:03.175449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 156/312] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Oct 4 10:07:02 2022] GET /identity/v3/role_assignments?user.id=cb4fadcd662c450795d835e82f59088c&scope.system=all&role.id=7ef4aa4b4180492cb81a40024c482a71 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:03.260671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:03.374744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128a361e-51e2-49a8-b0b1-738a164c89b2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:03.399048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:03.411446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:03.412371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:03.412947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:03.413594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:03.424028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:03.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:03.430803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:03.431828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 157/313] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:03 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:03.438918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:03.545560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08c0b4e-8905-4a79-94e4-18bc2a7d4e66 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:03.565610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:03.585579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:03.586191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:03.586495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:03.586811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:03.587482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] RBAC: Adding query filter params (name=reader) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:03.587783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:03.589402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:03.602339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 157/314] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:03.604267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:03.717332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efedd792-d8c1-4681-9f8c-e48fb0b331ac None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:03.752784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:03.753415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:03.754284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:03.754660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] PATH_INFO: `/v3/users/ee8a9fae99534e848c9f9db4dd776183` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:03.755989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] RBAC: Authorizing `identity:get_user(user_id=ee8a9fae99534e848c9f9db4dd776183)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:03.757431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:03.759230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 158/315] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:03 2022] GET /identity/v3/users/ee8a9fae99534e848c9f9db4dd776183 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:03.764820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:03.869665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66959605-9aed-4bb5-a5a5-6dfcb8d3bdda None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:03.906435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:03.907051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:03.907327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:03.907597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:03.964989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:03.990043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:03.991531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:04.103492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 158/316] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:04.104924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:04.166122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:04.221959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39d9bf-34b3-4ea6-9c42-fb307b6d272f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:04.258461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca8b93f-bfe8-4eaf-8832-3ccd2b080990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:04.259101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b93f-bfe8-4eaf-8832-3ccd2b080990 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:04.259379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b93f-bfe8-4eaf-8832-3ccd2b080990 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:04.259695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca8b93f-bfe8-4eaf-8832-3ccd2b080990 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:04.260822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8b93f-bfe8-4eaf-8832-3ccd2b080990 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:04.262271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca8b93f-bfe8-4eaf-8832-3ccd2b080990 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:04.263894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 159/317] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:04 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:04.270857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:04.354929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:04.378224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e713b830-f4fb-4831-8761-bcb633cd426c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:04.414980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:04.415588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:04.415864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:04.416186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:04.416864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] RBAC: Adding query filter params (role.id=7ef4aa4b4180492cb81a40024c482a71, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817, user.id=ee8a9fae99534e848c9f9db4dd776183) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:04.417172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:04.418650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd6d755-21de-47d7-a196-8a3b126d0d52 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:04.425216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 159/318] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:04 2022] GET /identity/v3/role_assignments?user.id=ee8a9fae99534e848c9f9db4dd776183&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=7ef4aa4b4180492cb81a40024c482a71 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:04.858430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322874-7e1b-4915-b63c-4c8d9a0e9bb8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:04.858782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322874-7e1b-4915-b63c-4c8d9a0e9bb8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:04.858985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25322874-7e1b-4915-b63c-4c8d9a0e9bb8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:04.860472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 160/319] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:04.866012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:04.866326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:04.866605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:04.884612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:04.885789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:04.954450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 160/320] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:05.222622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25322874-7e1b-4915-b63c-4c8d9a0e9bb8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:05.324587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25322874-7e1b-4915-b63c-4c8d9a0e9bb8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:05.342096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:05.361782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8014edb8-c13e-483e-9d40-a51864d153e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:05.362474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014edb8-c13e-483e-9d40-a51864d153e1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.362753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014edb8-c13e-483e-9d40-a51864d153e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.363038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8014edb8-c13e-483e-9d40-a51864d153e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:05.365015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014edb8-c13e-483e-9d40-a51864d153e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:05.366814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8014edb8-c13e-483e-9d40-a51864d153e1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:05.370253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 161/321] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:05.374315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:05.448092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:05.476432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced92214-d286-4654-aade-cbe4c1f77ee8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.476757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced92214-d286-4654-aade-cbe4c1f77ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.476902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced92214-d286-4654-aade-cbe4c1f77ee8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:05.478400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 162/322] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:05.481037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d457e3e-076f-4f87-91b2-d2b7a12c669a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:05.483289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.483577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.483838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:05.502254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:05.503430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:05.517766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0499794-8801-4301-a77b-a8e7d8fd39cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:05.518420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0499794-8801-4301-a77b-a8e7d8fd39cb None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.518696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0499794-8801-4301-a77b-a8e7d8fd39cb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.518964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0499794-8801-4301-a77b-a8e7d8fd39cb None admin] PATH_INFO: `/v3/groups` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:05.519756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0499794-8801-4301-a77b-a8e7d8fd39cb None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:05.521167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0499794-8801-4301-a77b-a8e7d8fd39cb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:05.545158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 161/323] 10.222.0.13 () {60 vars in 1134 bytes} [Tue Oct 4 10:07:05 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:07:05.545158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fbc0-fbfc-47a1-b54f-ea315a167401 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.545158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fbc0-fbfc-47a1-b54f-ea315a167401 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.545158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f91fbc0-fbfc-47a1-b54f-ea315a167401 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:05.545158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 162/324] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:05.548078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.548432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.549143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:05.570366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:05.571601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:05.576123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 163/325] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:05.652722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 163/326] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:05.843659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:05.933468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:05.951279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d9e579-6fd8-4c0c-a7d0-b1418d8f8aa4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:05.975399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:05.996656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:05.997609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:05.998056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:05.998535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.011773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:06.013964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:06.018019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:06.019307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:06.020122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:06.025171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 164/327] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:05 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:06.026628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:06.062273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cfd54a-4e67-4f7d-9477-ec8a0c31b42f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.108873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.109514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.109791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.110062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.124703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:06.131458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:06.132325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:06.133147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:06.133720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 164/328] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:06.137989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d230233d-8ea4-4c70-aab0-896f9df381db None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.139004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:06.186171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.186825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.187102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.187374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.188100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:06.188401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:06.189785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:06.201176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 165/329] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:06.207664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:06.260552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:06.262267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdcf1b-ea83-43f4-8659-6a007ee3dfde None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.293617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:06.331446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.332115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.332392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.332661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.333337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:06.333637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:06.335066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:06.337121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098589bd-7913-437d-aade-bf33ed647a1f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.353730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 165/330] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:06.363674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:06.383747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f699-536e-4666-a978-344f051d2598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.384854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f699-536e-4666-a978-344f051d2598 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.385138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f699-536e-4666-a978-344f051d2598 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.385416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df65f699-536e-4666-a978-344f051d2598 None admin] PATH_INFO: `/v3/users/cb4fadcd662c450795d835e82f59088c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.386647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f699-536e-4666-a978-344f051d2598 None admin] RBAC: Authorizing `identity:get_user(user_id=cb4fadcd662c450795d835e82f59088c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:06.388397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df65f699-536e-4666-a978-344f051d2598 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:06.390580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 166/331] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:06 2022] GET /identity/v3/users/cb4fadcd662c450795d835e82f59088c => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:06.397161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f699-536e-4666-a978-344f051d2598 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:06.498432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63df9a07-9717-424e-ab10-755c4f886f30 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.532956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df65f699-536e-4666-a978-344f051d2598 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.541744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:06.544075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.544718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.544994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.545264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] PATH_INFO: `/v3/users/308338bf4a8b4f7b907c62cc19910283` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.546491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] RBAC: Authorizing `identity:get_user(user_id=308338bf4a8b4f7b907c62cc19910283)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:06.547936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:06.549542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 166/332] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:06 2022] GET /identity/v3/users/308338bf4a8b4f7b907c62cc19910283 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:06.563193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:06.585560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed458e4b-f8d7-4c79-a514-879b6bcc156c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.586237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed458e4b-f8d7-4c79-a514-879b6bcc156c None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.586525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed458e4b-f8d7-4c79-a514-879b6bcc156c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.586803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed458e4b-f8d7-4c79-a514-879b6bcc156c None admin] PATH_INFO: `/v3/system/users/cb4fadcd662c450795d835e82f59088c/roles/7ef4aa4b4180492cb81a40024c482a71` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.588228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed458e4b-f8d7-4c79-a514-879b6bcc156c None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cb4fadcd662c450795d835e82f59088c, role_id=7ef4aa4b4180492cb81a40024c482a71)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:06.590052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed458e4b-f8d7-4c79-a514-879b6bcc156c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:06.603047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 167/333] 10.222.0.13 () {58 vars in 1334 bytes} [Tue Oct 4 10:07:06 2022] PUT /identity/v3/system/users/cb4fadcd662c450795d835e82f59088c/roles/7ef4aa4b4180492cb81a40024c482a71 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:07:06.641267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:06.703664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a0df98-c4a1-4002-ada0-b9582f407c04 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:06.747683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:06.748688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:06.749162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:06.749629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:06.750623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] RBAC: Adding query filter params (role.id=7c12d43981d3489c96be96ccaf741270, user.id=308338bf4a8b4f7b907c62cc19910283, scope.system=all) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:06.751226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:06.752960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9884619-5c8b-4257-9ce9-4e2ad961a8b7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:06.767396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 167/334] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Oct 4 10:07:06 2022] GET /identity/v3/role_assignments?user.id=308338bf4a8b4f7b907c62cc19910283&scope.system=all&role.id=7c12d43981d3489c96be96ccaf741270 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:08.290832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaf672-d499-44cf-9bea-d214089acf63 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:08.291326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaf672-d499-44cf-9bea-d214089acf63 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:08.291435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecaf672-d499-44cf-9bea-d214089acf63 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:08.292952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 168/335] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:08.299392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:08.299777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:08.299955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:08.319232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:08.320419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:08.395385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 168/336] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:08.679687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaf672-d499-44cf-9bea-d214089acf63 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:08.791663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecaf672-d499-44cf-9bea-d214089acf63 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:08.841181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08441a5-0032-45b1-9a7c-5b819b27a2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:08.842355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08441a5-0032-45b1-9a7c-5b819b27a2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:08.842859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08441a5-0032-45b1-9a7c-5b819b27a2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:08.843346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08441a5-0032-45b1-9a7c-5b819b27a2d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:08.861373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08441a5-0032-45b1-9a7c-5b819b27a2d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e08441a5-0032-45b1-9a7c-5b819b27a2d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:08.867238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:08.868086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:08.868859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:08.871138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 169/337] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:08.877157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:08.943686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:08.973743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd722-acd2-463d-b638-c56cc1d70bff None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:08.974451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd722-acd2-463d-b638-c56cc1d70bff None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:08.974932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458dd722-acd2-463d-b638-c56cc1d70bff None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:08.976801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 170/338] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:08.982607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:08.983188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:08.983663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:09.018396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:09.019986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:09.033880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7de10af-e927-4478-8dee-6bbe44185349 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:09.080383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:09.105808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:09.106467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:09.106750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:09.107022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:09.107703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:09.108008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:09.109395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:09.119092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 169/339] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:09.125118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:09.134138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 171/340] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:09.288381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb108d2-108f-4e54-9d17-4a72371ca220 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:09.289860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:09.333885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:09.334539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:09.334776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:09.335037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:09.427933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:09.429424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:09.454217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:09.564840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d1a92-8126-44ff-8a54-e3764be455d2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:09.580656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 170/341] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 10:07:09.587186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:09.589473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:09.674991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:09.676182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:09.676459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:09.676728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:09.721273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:09.730413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:09.731495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:09.732477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:09.733314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd723d1a-0efb-467e-9452-994a63bc35e8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:09.738751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 172/342] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:09.743210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:09.785533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:09.787468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:09.788495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:09.788921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:09.790618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:09.792120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:09.797551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:09.802004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 171/343] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:09 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:09.810289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:09.924351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f31bf-af04-455e-a67c-61af86a761e7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:09.942736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:09.981412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721fb14d-e6d4-4865-8c88-b189b66c4e85 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:09.982422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:09.983047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:09.983326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:09.983685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:09.984264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:09.984563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:09.986005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:09.999251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 173/344] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:10.011288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:10.038931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f3d5a5-ec31-4176-96b4-725e44e1e55a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:10.039905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d5a5-ec31-4176-96b4-725e44e1e55a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.040328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d5a5-ec31-4176-96b4-725e44e1e55a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.040741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f3d5a5-ec31-4176-96b4-725e44e1e55a None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.052723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3d5a5-ec31-4176-96b4-725e44e1e55a None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:10.054390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f3d5a5-ec31-4176-96b4-725e44e1e55a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:10.056563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 172/345] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:09 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:10.057988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b2029-036f-406a-ac3f-6eea73c814bc None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.058437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b2029-036f-406a-ac3f-6eea73c814bc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.058911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b2029-036f-406a-ac3f-6eea73c814bc None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.061103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 173/346] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:10.063683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b2029-036f-406a-ac3f-6eea73c814bc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:10.178158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:10.208764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c98df15-f4e9-4366-8085-be9dd2d8dedb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:10.278080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:10.279190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.279656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.280144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.282950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b2029-036f-406a-ac3f-6eea73c814bc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:10.336368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:10.338098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.339313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.339736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.340908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=6067280cdc4440acac78a21b0daba4da) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:10.341364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:10.343206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbfaba45-409c-46dd-ba77-e13cdc76689b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:10.346137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:10.353686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 174/347] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:10 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=7c12d43981d3489c96be96ccaf741270 => generated 260 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:10.355142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45bf33-c584-4682-b660-7fb44079db84 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.355569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45bf33-c584-4682-b660-7fb44079db84 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.355987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b45bf33-c584-4682-b660-7fb44079db84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.379599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:10.382233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:10.384019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0b45bf33-c584-4682-b660-7fb44079db84 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:10.385399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0b45bf33-c584-4682-b660-7fb44079db84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:10.544363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 175/348] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:10.545684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90cf8-ed09-4a29-abcc-5d99df77b417 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.545980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90cf8-ed09-4a29-abcc-5d99df77b417 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.550231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd90cf8-ed09-4a29-abcc-5d99df77b417 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.550231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 176/349] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:10.554312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7974-178d-49f6-a912-cbde4016bfac None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.554596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7974-178d-49f6-a912-cbde4016bfac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.554873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386d7974-178d-49f6-a912-cbde4016bfac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.562793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:10.588966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-386d7974-178d-49f6-a912-cbde4016bfac None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:10.590805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-386d7974-178d-49f6-a912-cbde4016bfac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:10.595637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 174/350] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 591 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 04 10:07:10.607880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:10.773691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 177/351] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:10.775030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c3db1-4d8f-4536-80be-fd46cd15cfe5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.775322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c3db1-4d8f-4536-80be-fd46cd15cfe5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.775582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167c3db1-4d8f-4536-80be-fd46cd15cfe5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.776861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 178/352] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:10.786589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.786876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.787138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.817007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb66f4c4-9b4f-4538-af8a-907754574677 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:10.833470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:10.834717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:10.903640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:10.904466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:10.904739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:10.905040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:10.906865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:10.909297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:10.918412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 175/353] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:10 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:10.927998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:10.966007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.004950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 179/354] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:11.006443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.038465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.141619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133dafd5-60fe-45b9-9f41-82663be76e03 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.172887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e33795e-f908-461d-b49c-c828d7f3e0a8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.191833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01b1a85-eac2-4edb-ab2b-9013c1f91b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.192805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1a85-eac2-4edb-ab2b-9013c1f91b87 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.193294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1a85-eac2-4edb-ab2b-9013c1f91b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.193752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01b1a85-eac2-4edb-ab2b-9013c1f91b87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.215653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01b1a85-eac2-4edb-ab2b-9013c1f91b87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a01b1a85-eac2-4edb-ab2b-9013c1f91b87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:11.222076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:11.223788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:11.224574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:11.228341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 180/355] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:11.230545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d0b50-76d2-4029-bd94-649bce8ffdfb None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.232267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d0b50-76d2-4029-bd94-649bce8ffdfb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.233456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d0b50-76d2-4029-bd94-649bce8ffdfb None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.235684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 181/356] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:11.237945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d0b50-76d2-4029-bd94-649bce8ffdfb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.246650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad3f585-1f4f-44db-9a79-fe28942a3050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.247569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3f585-1f4f-44db-9a79-fe28942a3050 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.248036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3f585-1f4f-44db-9a79-fe28942a3050 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.248511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad3f585-1f4f-44db-9a79-fe28942a3050 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.254802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3f585-1f4f-44db-9a79-fe28942a3050 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:11.256500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad3f585-1f4f-44db-9a79-fe28942a3050 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:11.258334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 176/357] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:10 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:11.259771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.260243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.260704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.280152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:11.281645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:11.330029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.370289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 177/358] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:11.372826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.401351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d0b50-76d2-4029-bd94-649bce8ffdfb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.435983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.439417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.440253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.440532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.441722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.442717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:11.443024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:11.445024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:11.456099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 182/359] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:11.457505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.498092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7ad0a4-f6fa-4118-b81a-98b4c4b84620 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.545742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.546377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.546653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.546923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.548100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=6067280cdc4440acac78a21b0daba4da) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:11.548408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:11.549830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:11.558614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 178/360] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 260 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:11.559310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.576996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0f1bc-4e8b-4323-aada-a3de1dbe4ed9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.589400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.618797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.619688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.620169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.620606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.637773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:11.642883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:11.643850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:11.645275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:11.648122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 183/361] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:11.649146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.687378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1d2cd7-a7bc-4faf-abd4-ee30aadfa065 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.704585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.735051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.735685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.735947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.736208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.750955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:11.758116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:11.759074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:11.759873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 179/362] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:11.760535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:11.776816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025bb89-1c04-4f5e-b3e6-770a87452641 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.817343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.818349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.818842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.819358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.865836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:11.892118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea41a5e-d9da-4136-9ad2-872335db391d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:11.905846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:11.907908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:11.936985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:11.937649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:11.937867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:11.938133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:11.938824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:11.939120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:11.940477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:11.952759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 180/363] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:11.954003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.005498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.043701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 184/364] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:12.045110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.095832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ff02d-fa54-4e9d-a2fd-5ee209750acd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.128120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.156549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.157513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.157987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.158497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.159355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:12.159897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.161995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.170751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 181/365] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.171937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.182716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7000874-d807-4a36-992c-88ff96756872 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.229967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.230811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.231093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.231364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.246102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:12.254253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:12.255346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 185/366] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:12.256158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.291962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.299083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ec53e9-1628-49f9-9d3c-2933bb8562d2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.344140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.345076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.345584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.345992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.390061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a70b938-f51f-4a44-b9d3-a8e38905981c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.429844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.444260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.445309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.446021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.446717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.447617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.449450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.451572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 186/367] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.452410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.462020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.463822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.604772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.605778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6643e9e7-7fd7-4064-8bc7-7200285baff8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.622331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 182/368] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:12.623384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.655553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.656184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.656457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.656723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] PATH_INFO: `/v3/users/cb4fadcd662c450795d835e82f59088c` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.657961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] RBAC: Authorizing `identity:get_user(user_id=cb4fadcd662c450795d835e82f59088c)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.659499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.661625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 187/369] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/users/cb4fadcd662c450795d835e82f59088c => generated 293 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.662531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.736786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.748343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed28e9-451e-4f1a-ba10-5988e10d3fe1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.775031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4379a4f-4341-467d-86bd-cfc2eeec6900 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.785910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.786316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.786550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.786818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.787477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:12.787835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.789260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.799223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 183/370] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.800064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.818834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.819567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.819844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.820218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.822320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.823971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.825561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 188/371] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.826449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.866041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.920761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69f8a8e-114b-4855-9c82-94e0811d6033 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.939539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75726781-b3fd-4ff5-af52-70e0873a41cc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:12.957350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.958127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.958432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.958707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.959855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.961425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.963403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 184/372] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.964353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:12.976062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:12.976664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:12.977032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:12.977303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:12.978031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] RBAC: Adding query filter params (role.id=7ef4aa4b4180492cb81a40024c482a71, user.id=cb4fadcd662c450795d835e82f59088c, scope.system=all) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:12.978379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:12.979892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:12.982658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:12.994973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 189/373] 10.222.0.13 () {56 vars in 1308 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/role_assignments?user.id=cb4fadcd662c450795d835e82f59088c&scope.system=all&role.id=7ef4aa4b4180492cb81a40024c482a71 => generated 510 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:12.996183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.080758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e76eb15-adaf-49cd-890f-fde2ee30d7bc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.113062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:13.117799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e933e7-3be9-4cd9-905a-cadedb05dfb5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.124784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a85e3c-47ae-4bf3-ad86-20e8e0f2688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:13.125427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a85e3c-47ae-4bf3-ad86-20e8e0f2688a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.125701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a85e3c-47ae-4bf3-ad86-20e8e0f2688a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.125974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a85e3c-47ae-4bf3-ad86-20e8e0f2688a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.160571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:13.161194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.161466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.162059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.162504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:13.162809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:13.164325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:13.171438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 190/374] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=7c12d43981d3489c96be96ccaf741270 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:13.172396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.216416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a85e3c-47ae-4bf3-ad86-20e8e0f2688a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:13.217937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a85e3c-47ae-4bf3-ad86-20e8e0f2688a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:13.268827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:13.346525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafa2be-e3d7-4676-b86f-73fb04b2de4c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.370162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 185/375] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:13.371923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bc742-dc20-47d1-9730-272ca698e25a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.372501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bc742-dc20-47d1-9730-272ca698e25a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.372787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bc742-dc20-47d1-9730-272ca698e25a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.374866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 186/376] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:13.375869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bc742-dc20-47d1-9730-272ca698e25a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.392454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb07cadb-6834-421a-ac6d-ecd4d2d109d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:13.393112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07cadb-6834-421a-ac6d-ecd4d2d109d6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.393394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07cadb-6834-421a-ac6d-ecd4d2d109d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.393668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb07cadb-6834-421a-ac6d-ecd4d2d109d6 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.395698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07cadb-6834-421a-ac6d-ecd4d2d109d6 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:13.397196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb07cadb-6834-421a-ac6d-ecd4d2d109d6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:13.398913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 191/377] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:13 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:13.400100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.402499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.402791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.414942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:13.427083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:13.429153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:13.510368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 192/378] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:13.511324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.515354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bc742-dc20-47d1-9730-272ca698e25a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.587736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:13.590353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.590632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.590956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] PATH_INFO: `/v3/projects/fe7a43104df8480199dd56c9a29fe00d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.592360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7a43104df8480199dd56c9a29fe00d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:13.593787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:13.598364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 187/379] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:13 2022] GET /identity/v3/projects/fe7a43104df8480199dd56c9a29fe00d => generated 344 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:13.603387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.634551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:13.709607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb7c713-0884-4b2d-907c-65602f5a2b77 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.729503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:13.744609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7b3d8e-b89f-4d34-bb12-5b8d9d1036e4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.777122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:13.777743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.778018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.778340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.779084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:13.779390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:13.780849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:13.794946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 193/380] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:13 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 260 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:13.807621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:13.808283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:13.808559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:13.808831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] PATH_INFO: `/v3/groups/3cc3bb34fc0f478ea80c3940ecaaf943` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:13.818121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] RBAC: Authorizing `identity:get_group(group_id=3cc3bb34fc0f478ea80c3940ecaaf943)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:13.819641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:13.821585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 188/381] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:13 2022] GET /identity/v3/groups/3cc3bb34fc0f478ea80c3940ecaaf943 => generated 229 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:13.827480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.953195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:13.960521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272378a9-1ef9-4f4e-b729-2990ed8a1ee3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:13.991697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:14.010636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:14.011612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.012088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.012550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.013525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] RBAC: Adding query filter params (group.id=3cc3bb34fc0f478ea80c3940ecaaf943, role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=fe7a43104df8480199dd56c9a29fe00d) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:14.014028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:14.015698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:14.022825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 194/382] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:13 2022] GET /identity/v3/role_assignments?group.id=3cc3bb34fc0f478ea80c3940ecaaf943&scope.project.id=fe7a43104df8480199dd56c9a29fe00d&role.id=e8cffbbc9b164ec5b458727785767845 => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:14.106617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36500270-7ceb-4eec-a2c2-685429bcb981 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:14.152150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:14.152796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.153073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.153346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.165529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:14.171101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:14.172039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:14.172822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 189/383] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:14.179897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:14.318288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c885693-f8c6-48a8-a9ca-9b7f798ac681 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:14.367780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:14.368393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.368667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.368933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.369606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:14.369904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:14.371429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2755e36-5f06-4b87-a617-6a76564adc09 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:14.387654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 195/384] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:14.393275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:14.435364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:14.514772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95624d-391f-422c-bd1d-3920eaacdef9 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.515269 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95624d-391f-422c-bd1d-3920eaacdef9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.515475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d95624d-391f-422c-bd1d-3920eaacdef9 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.516983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 196/385] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:14.522487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.522755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.523031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.550621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:14.552524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:14.558368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe70910-d4da-4b35-a057-90a9d9662a3e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:14.627482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:14.628421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:14.629266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.629552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.629879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.632843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:14.634196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:14.635808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 190/386] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:14 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:14.647723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:14.677744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 197/387] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:14.744215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:14.763111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f18b7f-e022-418f-9775-eb6d769e83b0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:14.799801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:14.801189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:14.801470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:14.801743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:14.802913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:14.804350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:14.806323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 191/388] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:14 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:14.811888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:14.966276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39f9ac8-899c-4e21-bd90-189ce5e9cb7c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:15.012398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:15.013064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:15.013339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:15.013610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da/groups/9daa247166f440a8a1b2a07403fb1c25/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:15.019545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] RBAC: Authorizing `identity:create_grant(project_id=6067280cdc4440acac78a21b0daba4da, group_id=9daa247166f440a8a1b2a07403fb1c25, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:15.021038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:15.048035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 198/389] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Oct 4 10:07:14 2022] PUT /identity/v3/projects/6067280cdc4440acac78a21b0daba4da/groups/9daa247166f440a8a1b2a07403fb1c25/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:15.120132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:15.246995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25bd90d-e9f2-494f-8292-fed59b49e5b9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:15.290446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:15.291409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:15.291859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:15.292303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:15.309211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:15.314794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:15.316374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:15.317347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:15.319507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 192/390] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:15.325580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:15.392212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:15.486877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b10321-ea99-4399-a5ff-82151b97fb36 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:15.533075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:15.533668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:15.533881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:15.534162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:15.535033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:15.535391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:15.536806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:15.550864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 199/391] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:15.557024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:15.588003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:15.716604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5fc902-ed6e-4940-9f8b-768d2aa8ea97 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:15.760027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:15.760682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:15.760958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:15.761231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:15.766966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:15.768416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:15.771073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 193/392] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:15 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:15.776888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:15.927243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:15.933083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2551ea3a-5172-4ba2-8414-54acbf0a3749 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:15.978951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033c3237-ccbc-4568-808e-b52ebc5c2f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:15.979590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c3237-ccbc-4568-808e-b52ebc5c2f89 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:15.979866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c3237-ccbc-4568-808e-b52ebc5c2f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:15.980137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033c3237-ccbc-4568-808e-b52ebc5c2f89 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:15.981257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c3237-ccbc-4568-808e-b52ebc5c2f89 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:15.982729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033c3237-ccbc-4568-808e-b52ebc5c2f89 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:15.984726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 200/393] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:15 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:15.997584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:16.078297 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba502a-3c62-42c9-a356-1e3db77e18b5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:16.078647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba502a-3c62-42c9-a356-1e3db77e18b5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:16.078853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba502a-3c62-42c9-a356-1e3db77e18b5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:16.080210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 201/394] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:16.086298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:16.086593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:16.086843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:16.115304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:16.115304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:16.115573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:16.145916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf4332-0859-4bd9-9f3f-119485296337 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:16.196424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 202/395] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:16.208706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d630e86-2384-4bd0-87b1-406999625f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:16.209738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d630e86-2384-4bd0-87b1-406999625f19 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:16.210165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d630e86-2384-4bd0-87b1-406999625f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:16.210613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d630e86-2384-4bd0-87b1-406999625f19 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da/groups/9daa247166f440a8a1b2a07403fb1c25/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:16.213025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d630e86-2384-4bd0-87b1-406999625f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=6067280cdc4440acac78a21b0daba4da, group_id=9daa247166f440a8a1b2a07403fb1c25, role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:16.214926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d630e86-2384-4bd0-87b1-406999625f19 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:16.231932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 194/396] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Oct 4 10:07:15 2022] PUT /identity/v3/projects/6067280cdc4440acac78a21b0daba4da/groups/9daa247166f440a8a1b2a07403fb1c25/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:07:16.631238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:16.757140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:16.765470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535b9e25-6ad9-48c9-8793-7834a390cfd5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:16.800566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:16.801320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:16.801594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:16.801862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:16.812487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:16.818122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:16.819273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:16.820435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 203/397] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:16.821265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6efbc-3afd-4277-b67f-6fe286f6b54a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:16.821563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6efbc-3afd-4277-b67f-6fe286f6b54a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:16.821844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a6efbc-3afd-4277-b67f-6fe286f6b54a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:16.826083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 195/398] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:16.827187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:16.828767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a34ed2-81d3-45b4-9ed8-90ba12b5a5b9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:16.829116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a34ed2-81d3-45b4-9ed8-90ba12b5a5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:16.829457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a34ed2-81d3-45b4-9ed8-90ba12b5a5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:16.867012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-79a34ed2-81d3-45b4-9ed8-90ba12b5a5b9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:16.868521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-79a34ed2-81d3-45b4-9ed8-90ba12b5a5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:16.949850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 196/399] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:16.999403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97962bd9-b13e-4e20-afba-2aedf51c5fd4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.048331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.048948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.049229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.049505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.052240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:17.052556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:17.053963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:17.055231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858642-3a25-4887-a1b3-ac3a1fe7221b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.055737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858642-3a25-4887-a1b3-ac3a1fe7221b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.056232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43858642-3a25-4887-a1b3-ac3a1fe7221b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.058010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 197/400] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:17.063044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.063558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.064044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.065712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 204/401] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:17.071281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.088418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:17.090354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:17.101098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:17.161774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 198/402] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:17.224044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aead5b4-9686-4ae9-96b6-6d9cd6895fb2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.259513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.260139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.260418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.260697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.261808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:17.263288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:17.264881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 205/403] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:17.271188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.273744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:17.368279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.380407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc22eb-e3dc-4a92-b28a-0f6d0043c1c3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.417368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.418010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.418323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.418605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.419741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:17.421301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:17.423083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 199/404] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:17.429362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.510005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:17.522615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:17.535732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce631ec-b2ee-4e7d-803e-e6555289f95d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.547210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8450db-fa0d-4ac5-9ade-549b2435cb59 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.585970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.587000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.587505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.587608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.588107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.588218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.588490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.588760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/groups/9daa247166f440a8a1b2a07403fb1c25/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.591042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e8d106e8b48f3be72ac35d6185817, group_id=9daa247166f440a8a1b2a07403fb1c25, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:17.592890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:17.604544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:17.610560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 200/405] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Oct 4 10:07:17 2022] PUT /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/groups/9daa247166f440a8a1b2a07403fb1c25/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:17.618391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:17.620157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:17.621089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:17.621879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:17.624650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 206/406] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:17.630328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.730040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cead8509-2dfc-4b25-a3b6-ac84999ec8e8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.766795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:17.772290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.772944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.773217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.773489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.788809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:17.794963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:17.795859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:17.796639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 201/407] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:17.804902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.852176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293ae6fe-50ae-429a-9d22-3fa23edac2d0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:17.900499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:17.901121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:17.901397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:17.901662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:17.902359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:17.902660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:17.904107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:17.917957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 207/408] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:17.922065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:17.938580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:17.986326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d0aaf-5e1a-4cb1-a5e1-3c40b8bb4d0a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.048378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.050191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.050843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.052490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.053357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:18.054699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.058936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.075822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 202/409] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:18.081909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55416ac7-664b-41b0-92cc-7085d65b322e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.083716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.133013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e393b39a-ab78-4327-842c-b329dfad1f5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.134051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393b39a-ab78-4327-842c-b329dfad1f5f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.135101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393b39a-ab78-4327-842c-b329dfad1f5f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.135609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e393b39a-ab78-4327-842c-b329dfad1f5f None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.137417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393b39a-ab78-4327-842c-b329dfad1f5f None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.139830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e393b39a-ab78-4327-842c-b329dfad1f5f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.142793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 208/410] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:17 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:18.144347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a854829-211a-4ec9-9ba5-0d6b37372a3c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.144973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a854829-211a-4ec9-9ba5-0d6b37372a3c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.145476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a854829-211a-4ec9-9ba5-0d6b37372a3c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.147314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 209/411] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:18.151683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a854829-211a-4ec9-9ba5-0d6b37372a3c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.158578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:18.215116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b210624-a4dc-4b7e-85d8-2f728ea58ec6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.251556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55105ad-484e-44b8-80e0-ddb4f39bab8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.252223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55105ad-484e-44b8-80e0-ddb4f39bab8d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.252499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55105ad-484e-44b8-80e0-ddb4f39bab8d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.252770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55105ad-484e-44b8-80e0-ddb4f39bab8d None admin] PATH_INFO: `/v3/groups/3cc3bb34fc0f478ea80c3940ecaaf943` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.254475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55105ad-484e-44b8-80e0-ddb4f39bab8d None admin] RBAC: Authorizing `identity:get_group(group_id=3cc3bb34fc0f478ea80c3940ecaaf943)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.255974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55105ad-484e-44b8-80e0-ddb4f39bab8d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.257140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:18.262231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 203/412] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:18 2022] GET /identity/v3/groups/3cc3bb34fc0f478ea80c3940ecaaf943 => generated 229 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:18.262231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.262231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.262231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.280226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:18.281413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:18.295840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a854829-211a-4ec9-9ba5-0d6b37372a3c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.333461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.334009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.334307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.334575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.335695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.337176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.339593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 210/413] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:18 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:18.340361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.383784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 204/414] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:18.384647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.446584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:18.500481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6b0b2-0475-4395-a25c-973a1d2474b1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.539180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64598eb7-0567-43dc-be56-a044883d7f93 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.539374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.539931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.540216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.540524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/groups/9daa247166f440a8a1b2a07403fb1c25/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.552401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] RBAC: Authorizing `identity:create_grant(project_id=4b1e8d106e8b48f3be72ac35d6185817, group_id=9daa247166f440a8a1b2a07403fb1c25, role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.553859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.576654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.577288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.577558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.577919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] PATH_INFO: `/v3/projects/fe7a43104df8480199dd56c9a29fe00d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.579252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] RBAC: Authorizing `identity:get_project(project_id=fe7a43104df8480199dd56c9a29fe00d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.580786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.583719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 211/415] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:18 2022] GET /identity/v3/projects/fe7a43104df8480199dd56c9a29fe00d => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:18.589435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.593476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 205/416] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Oct 4 10:07:18 2022] PUT /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817/groups/9daa247166f440a8a1b2a07403fb1c25/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:18.619873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:18.687030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.811312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0447aa5c-25fc-4ad5-b7aa-57215cb1df8f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.854487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a4c77d-836e-4986-9de3-8ff77b398f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.855217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4c77d-836e-4986-9de3-8ff77b398f62 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.855495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4c77d-836e-4986-9de3-8ff77b398f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.855815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a4c77d-836e-4986-9de3-8ff77b398f62 None admin] PATH_INFO: `/v3/projects/fe7a43104df8480199dd56c9a29fe00d/groups/3cc3bb34fc0f478ea80c3940ecaaf943/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.857731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4c77d-836e-4986-9de3-8ff77b398f62 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe7a43104df8480199dd56c9a29fe00d, group_id=3cc3bb34fc0f478ea80c3940ecaaf943, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:18.859247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a4c77d-836e-4986-9de3-8ff77b398f62 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:18.876223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b89e01-121e-43dd-a8cb-ddaea5bab139 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:18.885460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 212/417] 10.222.0.13 () {58 vars in 1442 bytes} [Tue Oct 4 10:07:18 2022] PUT /identity/v3/projects/fe7a43104df8480199dd56c9a29fe00d/groups/3cc3bb34fc0f478ea80c3940ecaaf943/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:18.900216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:18.932382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:18.933076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.933354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.933630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.942889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fff18-1d0d-4d88-acee-784b8367ed8c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.943229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fff18-1d0d-4d88-acee-784b8367ed8c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.943665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fff18-1d0d-4d88-acee-784b8367ed8c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.945521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 213/418] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:18.951292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:18.956385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:18.957351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:18.958114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 206/419] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:18.960517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:18.960517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:18.960517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:18.971287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:18.983659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:18.985262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:19.086734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:19.091347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 214/420] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:19.154912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0f1969-5e4d-4eb1-a13e-1dc97b516671 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:19.199127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:19.199745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:19.200024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:19.200296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:19.200958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:19.201257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:19.202692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:19.214763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 207/421] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:19.224007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:19.339514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bb3c7-e3bc-4f72-b178-a36c0e66ecce None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:19.374405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:19.375010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:19.375283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:19.375549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:19.443312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:19.450619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:19.452084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:19.517037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:19.571787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 215/422] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:19.577444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:19.592459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:19.717971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaccf669-9e2b-4796-8f99-ddfa0738560c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:19.752547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:19.753708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:19.754088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:19.754485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:19.766408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:19.772346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:19.773195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:19.773978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 208/423] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:19.781713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e3933f-0bb5-49e0-bf53-9981a00008f3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:19.784024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:19.784666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:19.863579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:19.864264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:19.864527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:19.864799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:19.868302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:19.869765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:19.871459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 216/424] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:19 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:19.877535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:19.969819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd92662-a597-4973-a813-51de54ca2ce7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:19.984881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:20.005108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:20.005728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.006184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.006485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.007150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:20.007453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:20.008843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:20.019324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 209/425] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:20.027336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:20.063191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aab597-cd17-4803-a7fa-43bfefdd1143 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:20.132635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:20.133270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.133547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.133826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.135770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:20.137212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:20.140001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 217/426] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:19 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:20.151201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:20.188036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebbd3a7-4ef1-40b8-839b-603592c11101 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:20.228816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:20.247337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:20.248275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.248704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.249157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.303191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:20.365405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9886f1c-4d81-4426-b895-40c482b6a91a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:20.388113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:20.390249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:20.404689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:20.405305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.405579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.406171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.407483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=6067280cdc4440acac78a21b0daba4da) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:20.408283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:20.411409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be56a9fe-108e-4e69-baae-e9302826bd0c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:20.419212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 218/427] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:20 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=7c12d43981d3489c96be96ccaf741270 => generated 611 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:20.426377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be170a-94d6-4ab7-8381-8c6e348d31e1 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.426377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be170a-94d6-4ab7-8381-8c6e348d31e1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.426377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be170a-94d6-4ab7-8381-8c6e348d31e1 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.426377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 219/428] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:20.433599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.433892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.434135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.469284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:20.470538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:20.570240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 210/429] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 543 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:20.575619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:20.614284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 220/430] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:20.691276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:20.744171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb6a413-e2b3-4358-9b64-f71561071216 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:20.782907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:20.783886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.785083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.785571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] PATH_INFO: `/v3/projects/6067280cdc4440acac78a21b0daba4da` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.786982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] RBAC: Authorizing `identity:get_project(project_id=6067280cdc4440acac78a21b0daba4da)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:20.788676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:20.790835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 211/431] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:20 2022] GET /identity/v3/projects/6067280cdc4440acac78a21b0daba4da => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:20.795969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:20.836588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:20.920862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caf6dd5-abff-46a0-9a1a-6a96eacc85a8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:20.932060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:20.974242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:20.976379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:20.976892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:20.977380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:20.978781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:20.980507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:20.983676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 221/432] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:20 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:20.989081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:21.072775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d8c9-465c-4630-97fa-b8d6366f2f72 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:21.101424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:21.121655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:21.122944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.123224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.123496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.130889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:21.136316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b70998-1554-4a51-a31b-5801c0cbb286 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:21.145658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:21.157043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:21.157970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 04 10:07:21.158807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:21.162134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 212/433] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:21.165476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:21.189302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:21.190320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.191586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.192711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.193744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=6067280cdc4440acac78a21b0daba4da) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:21.194306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:21.195948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532669b-4eef-4bbf-ade8-088aec283f89 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:21.204257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 222/434] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:20 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=6067280cdc4440acac78a21b0daba4da&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 611 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:21.247628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29fb01-837b-4880-bddd-3828766af3b0 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.247942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29fb01-837b-4880-bddd-3828766af3b0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.248173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba29fb01-837b-4880-bddd-3828766af3b0 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.251033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 223/435] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:21.256740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737289d7-7aa3-43a0-8973-12a1d0526eb2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.257052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737289d7-7aa3-43a0-8973-12a1d0526eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.257275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737289d7-7aa3-43a0-8973-12a1d0526eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.278833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-737289d7-7aa3-43a0-8973-12a1d0526eb2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:21.280141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-737289d7-7aa3-43a0-8973-12a1d0526eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:21.301518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6653d4d1-3526-496d-b48a-563327be353d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:21.363844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:21.365016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.365487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.366017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.366876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] RBAC: Adding query filter params (name=member) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:21.367403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:21.369010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:21.381051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 213/436] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:07:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:21.387520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:21.394380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 224/437] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:21.437093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:21.439674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54872c-20b6-499e-8ec7-c0589962f06e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.440177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54872c-20b6-499e-8ec7-c0589962f06e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.440741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b54872c-20b6-499e-8ec7-c0589962f06e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.448300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 225/438] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:21.450671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.451087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.451428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.482413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:21.484331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:21.502707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0093a-dbc3-4025-ba09-fcdf9763156e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:21.541121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:21.541733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.542004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.542307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.594060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 226/439] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:21.630347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:21.630347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:21.676379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:21.735798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 214/440] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:21.741951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:21.742193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:21.773747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:21.857556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33371cc-c74e-47de-94b8-3165a0b0ae84 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:21.881393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e67281-4c29-4001-822f-2a6119bff941 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:21.897519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:21.898157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.898471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.898773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.899907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:21.901319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:21.902952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 215/441] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:21 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:21.903693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:21.916326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:21.916951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:21.917229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:21.917495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:21.918268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:21.928429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:21.933271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:21.934115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:21.934923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 227/442] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:21.935563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.003995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99f1789-a929-4e82-8646-33206e7dc022 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.014509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.032686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d32da-5100-425d-9727-3b4efa76df82 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.036884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.037502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.037775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.038043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.047841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:22.052844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:22.053901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 216/443] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:22.054891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.070545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.071168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.071442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.071710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.072802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.074253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.075843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 228/444] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:21 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:22.076588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.127363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.157250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5adefa4-3110-4cd9-ac3b-4ce6c3325832 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.176362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a0dac7-357a-4c80-8f3b-c93449458804 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.195074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.195679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.195959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.196232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.196887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:22.197186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.198585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.206894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 217/445] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:22.207698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.215318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.216134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.216588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.216991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.217693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:22.218033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.219472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.228334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 229/446] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:22.229111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.243241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.311362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11d2038-1587-4c04-a78e-df44c1115c3a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.326576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3eca6b-64b4-4e79-a52f-625968495828 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.344421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.345051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.345338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.345656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.346422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=7c12d43981d3489c96be96ccaf741270, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:22.346822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.348365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.350198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.354764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 218/447] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=7c12d43981d3489c96be96ccaf741270 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:22.355669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.361039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.361941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.362921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.363335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.439890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.442508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.458783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.473965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c7f34-2d1d-412f-ace2-9da034516c60 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.511331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.512295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.512760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.513225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.561653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 230/448] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:22.567603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.588725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.595044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.598521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.687816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dda77c-12cb-4cba-94db-d489509f47f7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.702227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.718252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 219/449] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:22.724579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.730959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.731597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.731874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.732144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] PATH_INFO: `/v3/projects/4b1e8d106e8b48f3be72ac35d6185817` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.733262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] RBAC: Authorizing `identity:get_project(project_id=4b1e8d106e8b48f3be72ac35d6185817)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.734736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.736365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 231/450] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/projects/4b1e8d106e8b48f3be72ac35d6185817 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:22.742197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.823892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.826816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2dae0d-bded-4343-8321-ec0b3009e210 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.840630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eaef7d-d380-4ff9-9528-80e0b30c8591 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.860746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.861368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.861645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.861917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] PATH_INFO: `/v3/projects/fe7a43104df8480199dd56c9a29fe00d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.863650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] RBAC: Authorizing `identity:get_project(project_id=fe7a43104df8480199dd56c9a29fe00d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.865062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.866914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 220/451] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/projects/fe7a43104df8480199dd56c9a29fe00d => generated 344 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:22.872179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.874136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:22.874787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:22.875065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:22.875337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] PATH_INFO: `/v3/groups/9daa247166f440a8a1b2a07403fb1c25` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:22.876443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] RBAC: Authorizing `identity:get_group(group_id=9daa247166f440a8a1b2a07403fb1c25)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:22.877922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:22.879497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 232/452] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/groups/9daa247166f440a8a1b2a07403fb1c25 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:22.884960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:22.933040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:22.970013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d72c6c-a430-4289-94c8-167e88ef0d10 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:22.982726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b9fc76-dced-4ef0-9b4a-931b0d8211c3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:23.002044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:23.002701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:23.003030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:23.003300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] PATH_INFO: `/v3/groups/3cc3bb34fc0f478ea80c3940ecaaf943` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:23.004421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] RBAC: Authorizing `identity:get_group(group_id=3cc3bb34fc0f478ea80c3940ecaaf943)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:23.005838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:23.007428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 221/453] 10.222.0.13 () {56 vars in 1180 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/groups/3cc3bb34fc0f478ea80c3940ecaaf943 => generated 229 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:23.013236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:23.015544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:23.016143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:23.016420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:23.016690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:23.017411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] RBAC: Adding query filter params (group.id=9daa247166f440a8a1b2a07403fb1c25, role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=4b1e8d106e8b48f3be72ac35d6185817) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:23.017715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:23.019881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51450182-a610-4d8d-9232-ad6ffc6e67f3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:23.025565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 233/454] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:22 2022] GET /identity/v3/role_assignments?group.id=9daa247166f440a8a1b2a07403fb1c25&scope.project.id=4b1e8d106e8b48f3be72ac35d6185817&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:23.048298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:23.118608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd5812-9c40-43d4-935e-00df2fd7ec3e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:23.153529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:23.154138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:23.154444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:23.154716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:23.155448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] RBAC: Adding query filter params (group.id=3cc3bb34fc0f478ea80c3940ecaaf943, role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=fe7a43104df8480199dd56c9a29fe00d) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:23.155752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:23.157184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76e1adf-f131-4ff3-b32e-051f0f108c78 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:23.164769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 222/455] 10.222.0.13 () {56 vars in 1376 bytes} [Tue Oct 4 10:07:23 2022] GET /identity/v3/role_assignments?group.id=3cc3bb34fc0f478ea80c3940ecaaf943&scope.project.id=fe7a43104df8480199dd56c9a29fe00d&role.id=e8cffbbc9b164ec5b458727785767845 => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:26.051916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6f383-d796-4cd6-9870-bf57df043137 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.052940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6f383-d796-4cd6-9870-bf57df043137 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.053431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b6f383-d796-4cd6-9870-bf57df043137 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.055392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 234/456] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:26.061401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640be8c3-445f-4721-82e7-094dc114ff50 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.062309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640be8c3-445f-4721-82e7-094dc114ff50 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.062844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640be8c3-445f-4721-82e7-094dc114ff50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.091212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-640be8c3-445f-4721-82e7-094dc114ff50 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:26.092421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-640be8c3-445f-4721-82e7-094dc114ff50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:26.176145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf1d33-c9b5-485c-bd2d-12270ff31680 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.176415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf1d33-c9b5-485c-bd2d-12270ff31680 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.176854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf1d33-c9b5-485c-bd2d-12270ff31680 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.178997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 235/457] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:26.185181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a6e1-cc0c-4eef-bc7e-33b19ee1016f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.185788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a6e1-cc0c-4eef-bc7e-33b19ee1016f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.186304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9a6e1-cc0c-4eef-bc7e-33b19ee1016f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.195879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 223/458] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:26.223839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fac9a6e1-cc0c-4eef-bc7e-33b19ee1016f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:26.225300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fac9a6e1-cc0c-4eef-bc7e-33b19ee1016f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:26.321076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 236/459] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:26.401764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0049b-e92d-428c-9808-37cb3ca2a490 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.402306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0049b-e92d-428c-9808-37cb3ca2a490 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.402746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e0049b-e92d-428c-9808-37cb3ca2a490 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.404248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 224/460] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:26.410442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.410944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.411361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.439527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:26.442502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:26.444452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d20b3-4527-4e40-819e-b5e50c2d2dc5 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.444914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d20b3-4527-4e40-819e-b5e50c2d2dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.445350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d20b3-4527-4e40-819e-b5e50c2d2dc5 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.447112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 225/461] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:26.452919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.453478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.453978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.476204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:26.477730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:26.512444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:26.513300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:26.577108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 237/462] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:26.597084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 226/463] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:26.610580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:26.676179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:26.731653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f965f-aa2a-429b-b810-d3659940dedb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:26.790825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:26.791899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.792804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.813443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:26.814940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:26.817205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9883a17-ea96-4713-abe1-900d678c30c4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:26.845893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:26.846935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:26.847867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:26.847867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:26.847867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:26.855346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 238/464] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:26.856229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:26.886983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:26.887676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:26.887952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:26.888246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:26.899490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:26.903010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:26.904550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:26.920471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:26.921392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:26.922290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:26.922290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:26.922290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:26.923239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 227/465] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:26.924788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:26.938457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:26.990977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f0742b-3b5d-4df3-8b01-57657d6aca58 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.029273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.029877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.030155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.030452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.030765 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bb1e05-e1e4-4af9-82f0-e853c894cd0d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.031094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:27.031389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.032995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.045128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 239/466] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:27.045947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.072456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.073093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.073371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.073646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.084650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.086080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:27.100265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:27.101147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:27.101984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:27.101984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:27.101984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:27.101984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 228/467] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:27.102215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.142043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:27.160679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b8f51-772a-4e86-bf2e-196bfe8af9ba None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.164802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:27.198427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.199058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.199334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.199604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.203682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc041663-8e2f-45e9-881b-85b4020f82a8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.210088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.211702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:27.228161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:27.229113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:27.229922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:27.229922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:27.229922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:27.229922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 240/468] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:27.230138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.245157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.245828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.246040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.246344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.247013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:27.247280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.248696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.265849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 229/469] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:27.265849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.341716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738e497-41a5-4c45-bb1b-246c5fab5187 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.376232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.376556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006b20bf-23e3-43aa-be27-fcedf895c69d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.376840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.377117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.377387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.378685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.380130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.389228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:27.403586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:27.413679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 241/470] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:07:27 2022] POST /identity/v3/users => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:27.413679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.427161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.427813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.428091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.428370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.429021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:27.429323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.430807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.443829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 230/471] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:27.444689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.526095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71c837-0ad0-48fd-9a4d-d5b9d456b62f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.561583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa11ed2-8a3e-4e5e-92a4-94b7a9f1903c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.564123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.564746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.565049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.565915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.566831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:27.567158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.568559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd1d490-003b-4aae-8681-f16b6dad259f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.581483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 242/472] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:27.582784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f8a0c-e860-421a-82bf-5a89840b1f5c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.583061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f8a0c-e860-421a-82bf-5a89840b1f5c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.583332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716f8a0c-e860-421a-82bf-5a89840b1f5c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.584551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 243/473] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:27.589521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f8a0c-e860-421a-82bf-5a89840b1f5c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.609805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.610450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.610732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.611002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.611648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.613621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.645069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 231/474] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:07:27 2022] POST /identity/v3/users => generated 248 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:27.646218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:27.658566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:27.703994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:27.722170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716f8a0c-e860-421a-82bf-5a89840b1f5c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.765138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22120a43-7d00-4d2e-9544-49f330212397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.766324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22120a43-7d00-4d2e-9544-49f330212397 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.766864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22120a43-7d00-4d2e-9544-49f330212397 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.767408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22120a43-7d00-4d2e-9544-49f330212397 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.769185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22120a43-7d00-4d2e-9544-49f330212397 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.771102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22120a43-7d00-4d2e-9544-49f330212397 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.794485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddeba63-377c-49d5-8a86-aff3ce138cfe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:27.798733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 244/475] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:07:27 2022] POST /identity/v3/users => generated 249 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:27.800562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.801059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.801613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.833704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:27.834987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:27.849984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:27.850994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:27.851412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:27.851819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:27.852618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:27.854196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:27.881424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 232/476] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:07:27 2022] POST /identity/v3/users => generated 248 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:27.925110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 245/477] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:28.210838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:28.317643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:28.324099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a40a9-5f46-4f92-a44e-c03cc10155f1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:28.373184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:28.373797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:28.374074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:28.374373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:28.375098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:28.376458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:28.387307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.sql.core [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 10:07:28.387307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 10:07:28.387307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [parameters: {'id': '3f5249f354164f2595a631807f615e08', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 10:07:28.387307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 04 10:07:28.405879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return ref.to_dict() Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:07:28.407541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application yield resource Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise self.value Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application session.commit() Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 10:07:28.409815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.flush() Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application compat.raise_( Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 10:07:28.413358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:07:28.416512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:07:28.423709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [parameters: {'id': '3f5249f354164f2595a631807f615e08', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:28.435320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 04 10:07:28.438461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:28.441530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 233/478] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Oct 4 10:07:28 2022] POST /identity/v3/roles => generated 144 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:28.441744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:28.508293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:28.607325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca377bb1-db45-401f-96ab-abbcd6ceb992 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:28.671656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bfb424-fee0-4a2a-969f-ca3d2f1b2526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:28.672294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfb424-fee0-4a2a-969f-ca3d2f1b2526 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:28.672573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfb424-fee0-4a2a-969f-ca3d2f1b2526 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:28.672844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bfb424-fee0-4a2a-969f-ca3d2f1b2526 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:28.685636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bfb424-fee0-4a2a-969f-ca3d2f1b2526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-06bfb424-fee0-4a2a-969f-ca3d2f1b2526 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:28.691016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:28.691968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:28.692962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 246/479] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:07:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:28.699165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:28.818165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d517fa4f-7700-4649-88a8-7df0054a21d7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:28.853137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:28.853750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:28.854029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:28.854376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:28.855047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:28.855346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:28.856722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf72e3e9-4c38-44c0-82f0-e4d3fa32eeae None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:28.866081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 234/480] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:29.938890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43cdf-22be-43a1-8fbd-8f8494c6abee None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:29.939542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43cdf-22be-43a1-8fbd-8f8494c6abee None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:29.939542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e43cdf-22be-43a1-8fbd-8f8494c6abee None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:29.941195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 247/481] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:29.946220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:29.946679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:29.946826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:29.971334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:29.972714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:29.987778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:30.060615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 235/482] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:30.461873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e43cdf-22be-43a1-8fbd-8f8494c6abee None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:30.596337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:30.602810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e43cdf-22be-43a1-8fbd-8f8494c6abee None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:30.650122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafa841-eba1-4e52-aa1d-d9c10dd26cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:30.650795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafa841-eba1-4e52-aa1d-d9c10dd26cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:30.651068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafa841-eba1-4e52-aa1d-d9c10dd26cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:30.651332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafa841-eba1-4e52-aa1d-d9c10dd26cf9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:30.666549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafa841-eba1-4e52-aa1d-d9c10dd26cf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2dafa841-eba1-4e52-aa1d-d9c10dd26cf9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:30.671468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:30.672313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:30.673079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 248/483] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:30.679476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:30.749652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abd00c-8f4f-4e5e-ac44-6e7caa9cddc3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:30.749959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abd00c-8f4f-4e5e-ac44-6e7caa9cddc3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:30.750121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97abd00c-8f4f-4e5e-ac44-6e7caa9cddc3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:30.751534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 249/484] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:30.757991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b0773-b402-488f-aaed-414db4bc9155 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:30.758290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b0773-b402-488f-aaed-414db4bc9155 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:30.758566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57b0773-b402-488f-aaed-414db4bc9155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:30.785724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e57b0773-b402-488f-aaed-414db4bc9155 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:30.787063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e57b0773-b402-488f-aaed-414db4bc9155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:30.835693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0de25-a214-44f7-af03-1e1c8ef83162 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:30.874074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:30.874948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:30.875372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:30.875776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:30.876577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:30.877010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:30.879230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:30.885676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 250/485] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Oct 04 10:07:30.887225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a52d6-cc55-44fa-85a3-3e6532e595b8 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:30.887511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a52d6-cc55-44fa-85a3-3e6532e595b8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:30.887781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219a52d6-cc55-44fa-85a3-3e6532e595b8 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:30.888810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 236/486] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:30.898328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 251/487] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:30.898507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:30.905582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80244c23-9ef8-4080-bf90-0051780852d9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:30.906021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80244c23-9ef8-4080-bf90-0051780852d9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:30.906451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80244c23-9ef8-4080-bf90-0051780852d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:30.929265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-80244c23-9ef8-4080-bf90-0051780852d9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:30.929836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:30.931846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-80244c23-9ef8-4080-bf90-0051780852d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:31.012332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 252/488] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:31.018818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb39528-9142-41c3-a6a2-871ff32f13fb None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.019105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb39528-9142-41c3-a6a2-871ff32f13fb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.019608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb39528-9142-41c3-a6a2-871ff32f13fb None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.020957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 253/489] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:31.022725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf297849-ebf4-4fc0-9418-8d70c2c63a25 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.026314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384305a-4567-4596-8bbe-5252df39462b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.026314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384305a-4567-4596-8bbe-5252df39462b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.026511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3384305a-4567-4596-8bbe-5252df39462b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.055556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3384305a-4567-4596-8bbe-5252df39462b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:31.056860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3384305a-4567-4596-8bbe-5252df39462b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:31.063945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.065638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.066251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.066733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.071683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.092240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:31.094054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:31.117667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:31.118618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:31.119479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:31.119479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:31.119479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:31.125847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 237/490] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:31.127314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.131644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 254/491] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:31.194758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.204167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384305a-4567-4596-8bbe-5252df39462b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.238267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e83650-a559-4c9d-a3ef-f69d67f1984e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.271082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.271082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.274546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.274837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.275493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:31.276063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:31.277154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:31.308085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 238/492] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:31.320090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.368076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3384305a-4567-4596-8bbe-5252df39462b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.447166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.447937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.448212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.448481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.462169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:31.467260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:31.468148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:31.468942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 255/493] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:31.469612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.477451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9144d4-f4e3-44d6-8852-3773e0439b04 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.481988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.506768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.517573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.519439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.519729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.520004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.528160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:31.529442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:31.543315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:31.544204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 239/494] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/users/nova => generated 82 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:31.545177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.591683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a996ff9f-fe80-4f53-a2ea-4b821439aa59 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.637346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.637749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.638002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.638352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.649412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:31.654071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:31.654981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:31.655770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 256/495] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:31.656413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.666481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6236a-b0d8-4552-8244-09fe0734234e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.702339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.702964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.703236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.703506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.713339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:31.719532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:31.720379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:31.721701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 240/496] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:31.722633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.730746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.768803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86082d-6143-457d-958c-e93023e660dd None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.772235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.802807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.803347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.803625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.803915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.804557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:31.804860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:31.806256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:31.813984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 257/497] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:31.815299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.826611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5737f18-5c88-41f7-b24d-ba4ccdaca807 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.861102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.862185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.862655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.862931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.863652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:31.863949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:31.865383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c22d150-f176-4bd7-ada3-bb6895d70e1b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:31.900913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 241/498] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:31.901980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c1a9d-8460-4dae-afcf-97a7de5b0b92 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.902291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c1a9d-8460-4dae-afcf-97a7de5b0b92 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.902564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3c1a9d-8460-4dae-afcf-97a7de5b0b92 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.903926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 242/499] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:31.904418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c1a9d-8460-4dae-afcf-97a7de5b0b92 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:31.916468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317bc50-a68a-4200-839f-e729e51a5df9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:31.945719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:31.950991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:31.951589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:31.951863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:31.952131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:31.952788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:31.953086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:31.954465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:31.962271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 258/500] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:31.963048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.008804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3c1a9d-8460-4dae-afcf-97a7de5b0b92 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.010417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.045015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.045702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.045872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.046138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.046832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:32.047135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.048501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.056429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 243/501] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:32.056948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.062122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d9acd-b373-4892-aa65-924d0b98af29 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.097417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.098042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.098354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.098632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.108894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.110372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-16612ec9-fa87-4b53-b595-549f0c8bcad9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:32.124570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:32.125494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:32.126348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:32.126348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.126348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:32.126348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 259/502] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:32.127024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.127024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.127318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.145965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:32.147192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:32.155147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba3554f-7ed5-4fc9-a4ce-659c82ef3fb7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.187572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.188099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.188694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.188967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.189242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.219874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.222425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 260/503] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:32.223704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.258773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.260217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.323962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47b23-0125-4126-877c-950b1cb6f8a9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.360526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.361153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.361425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.361785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.366849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 244/504] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:32.366849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.376352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.378447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:32.392950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:32.394463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:32.395314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:32.395314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.395314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:32.399385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 261/505] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:32.400083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.426289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.431442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.471458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d5909-52fb-406e-b74a-de4e6de93bc6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.500555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb73cdf-15cf-42db-91ff-e006c2abbdda None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.506407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.507034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.507308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.507579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.517575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.519176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.532860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:32.533032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:32.534536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:32.535377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:32.535377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.535377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:32.535538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.535538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.535538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.535726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 245/506] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:32.535726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.535842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:32.535905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.537244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.548743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 262/507] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:32.550153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.632135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.635103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab3b314-104e-4cd9-8f8c-8ec05cba002d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.648570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48396a03-5ecd-4e74-8f72-88a44d39fbce None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.651468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.669449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.670069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.670430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.670704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.681117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.682411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.682589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.683011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.683284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.683549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.684187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:32.684488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.685826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.696547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 263/508] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:32.696661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:32.697477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:32.698326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:32.698326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.698326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:32.698498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.698567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 246/509] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:32.699114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.796506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc35f4f6-2133-40cb-8e0e-5d058cda4ed3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.797620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0f7fa1-9e48-4381-9cdb-e456ef5113ab None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.829330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.829920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.830193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.830515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.831144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:32.831255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.831440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.831765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.832042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.832307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.832799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.843222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.844451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 247/510] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:32.844641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.845243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.851390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:32.859830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:32.860740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:32.862124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:32.862124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:32.862124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:32.862124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 264/511] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:32.862124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:32.875738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:32.949766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634d37c-8384-4f44-a854-eed2939fa184 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.961064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f47e47-f59f-4ce1-a6f7-1ed81e28c6c9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:32.983573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.984191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.984462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.984736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:32.992825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:32.994269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:32.995298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:32.995917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:32.996190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:32.996460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] PATH_INFO: `/v3/users/glance` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.004968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:33.005582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:33.006528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:07:33.007557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:33.008314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.008381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 248/512] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:33.008381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:33.023212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:33.024224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 265/513] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:32 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:33.025124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.086196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.104950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.117176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49c0f85-1ac3-45d6-a9d6-db9a5bc4a228 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.128279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d508fce-def6-442a-9345-da95d09788bc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.151407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.152014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.152288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.152564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.153196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:33.153496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.154916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.160777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.161387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.161658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.161921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.167094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 249/514] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:33.168143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.170319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.171998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:33.183314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:33.184183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:33.185035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 266/515] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/users/cinder => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:33.185588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.270977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecbb46f-924d-492c-94c4-7c9ecc142db2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.285741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329809f-e949-42eb-a76a-da160059af88 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.302112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.306128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.306777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.307052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.307324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.307958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:33.308260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.309611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.318483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.319072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.319343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.319608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.320318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:33.320609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 250/516] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:33.320695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.321392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.322021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.332054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.360004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 267/517] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:33.360840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.422348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebadea5-6100-40b4-a82a-96d68a5f2cca None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.455039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.456019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d61c6a1-75bd-4508-8dc5-0d9f720f0bbf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.456213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.456573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.456850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.457571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:33.457873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.459337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.476139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 251/518] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:33.476941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.489890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.490537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.490809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.491072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.497836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.499335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:33.511049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:33.512069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:33.512937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 268/519] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:33.513407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.547470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.549896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.584191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43105a8-35a8-4255-94e4-99990ef91bda None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.611934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d625-4464-43e9-bfcf-3adada994d57 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.617305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.617914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.618189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.618485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.619215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:33.619518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.620943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.637977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 252/520] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:33.638949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.649897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.650529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.650804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.651070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.651701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.653230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.673803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 269/521] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:07:33 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:33.674771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:33.758607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200bd1fc-82d6-4097-8f7b-646615faebe6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.779678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.788464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:33.791835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6201371-ce53-479b-a8a4-ab11eef42ab5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:33.800555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.801504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.801504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.801671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.833268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:33.833905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:33.834352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:33.834486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:33.889009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.891170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:33.915894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:33.917350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.004605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 253/522] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:34.005807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.023968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.033238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.040776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 270/523] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:34.042127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.112559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffa7d0-618c-46cf-b622-229351eef18f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.149318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.149919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.150193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.150337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de621a7d-dda8-425c-b3bb-fcee08693af2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.150740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.151159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:34.151457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.153829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.178509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 254/524] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:34.178509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.191238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.191843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.192116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.192380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.259960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.268544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.272972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.274742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.287562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259929b-4e66-4d8c-ba3c-bb4bdf19d59e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.321932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.322588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.322868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.323138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.333568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.335050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:34.350292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:34.351266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:34.352080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:34.352080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:34.352080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:34.352080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 255/525] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:34.352303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.400622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 271/526] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:34.400622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.464824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c568394-2132-4d44-8985-c5fa416eee2f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.496363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.504172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.504800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.505074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.505344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.506575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9092d1-7aea-40d2-a4d4-a8201a1aa924 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.508302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.517368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.519403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:34.535434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:34.536364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:34.537274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:34.537274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:34.537274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:34.537274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 256/527] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:34.537875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.547790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.548389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.548869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.548973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.549667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=a8514e96615c4785aefde8261120afe3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:34.549968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.551438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.556772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 272/528] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/role_assignments?user.id=a8514e96615c4785aefde8261120afe3&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:34.557535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.659834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0769ecf4-67af-4178-9743-d1b8f6351063 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.701132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.701711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.701986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.702304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.702964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:34.703277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.704845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.706132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9f0946-98f9-41c2-a7c1-c6d057080c74 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.722791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 257/529] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:34.723764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.755631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.757211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.757689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.758367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.778410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.780398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.781369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:34.804894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:34.806029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:34.807086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:34.807086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:34.807086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:34.808806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 273/530] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:34.809734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.829826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:34.859188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fdf5f-a6e4-445f-bf09-b5210dc26764 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.897273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.897977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.898341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.898674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:34.899480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:34.899848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:34.901388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:34.914727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 258/531] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:34.915711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:34.950805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465775b-2172-4119-a8a8-cf0b9943f937 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:34.993261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:34.994263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:34.994863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:34.996081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.004719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.007080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:35.022382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:35.023449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:35.024467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:35.026881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 274/532] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:35.028534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.030327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d517166-c971-40f8-ad9e-0b9286de3c6c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.040650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:35.070525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.071366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.071737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.072106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.073228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.073228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.074826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.089523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 259/533] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:35.090660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.111441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:35.155318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc4ba5d-52a1-4caf-be40-c6fa3c39645f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.194158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.194883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.195279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.195721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.203502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.205189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.214553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b19220-8687-4908-8b89-09570dcf62c1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:35.222194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:35.223102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:35.223968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 275/534] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:35.225640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.263440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.264047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.264322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.264591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.265273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.265570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.267283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.286183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 260/535] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:35.287086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.318720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:35.350853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c26ba9-02ec-43c3-b284-35baf8f34b40 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.386953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:35.397346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.398328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.398676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.399164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.406449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ec5db2-f69e-48ff-b530-1ed197fce8db None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.407987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.411217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:35.425803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:35.426845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:35.427669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:35.429898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 276/536] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:35.430993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.451654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.452589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.453058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.453534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.454436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.454930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.456685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.474336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 261/537] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:35.475474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.544459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e0a569-876f-4a80-88a1-0c694fa5416e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.583350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.584317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.585415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:35.586072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.586644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.588086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=0e568115c67243d3904359b84e0a790e) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.588654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.591983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.599578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 277/538] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/role_assignments?user.id=0e568115c67243d3904359b84e0a790e&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:35.600522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.601310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ff4c98-dc29-4e41-a653-5a77b82af1d3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.644755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.645612 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.646018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.646454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.647275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.647777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.649326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.664168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 262/539] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:35.670248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:35.671847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:35.775336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529c0142-12ca-479d-baef-02caa8a583ab None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.815511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.816380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.816795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.817228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.818110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=d5c6c66598c74c11a1c9cac119644c19) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.818581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.820476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd9d1d-e817-4b0f-aa25-8a32eb881756 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.827785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 278/540] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/role_assignments?user.id=d5c6c66598c74c11a1c9cac119644c19&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:35.884469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cb486-11f5-4187-9523-905c7f7f339a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:35.926664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:35.927289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:35.927569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:35.927842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:35.928577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=99de0be14e684b3696a4570987119ea7) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:35.928885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:35.930359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3aee3f-c43e-471c-bd17-b0f4f9924535 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:35.942788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 263/541] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:35 2022] GET /identity/v3/role_assignments?user.id=99de0be14e684b3696a4570987119ea7&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:36.062279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d7b7f-13e9-4040-ae1f-c7bfab399b2e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:36.062919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d7b7f-13e9-4040-ae1f-c7bfab399b2e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:36.063388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645d7b7f-13e9-4040-ae1f-c7bfab399b2e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:36.066006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 279/542] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:36.072059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7a13-bd04-4280-9898-52ea0666c7e9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:36.072342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7a13-bd04-4280-9898-52ea0666c7e9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:36.072620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b7a13-bd04-4280-9898-52ea0666c7e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:36.107380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4b0b7a13-bd04-4280-9898-52ea0666c7e9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:36.108627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4b0b7a13-bd04-4280-9898-52ea0666c7e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:36.117825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:36.192490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 264/543] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:36.898532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d7b7f-13e9-4040-ae1f-c7bfab399b2e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.015962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff101d-1af2-41c9-8509-e5c68ed4fc29 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.017074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff101d-1af2-41c9-8509-e5c68ed4fc29 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.017559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ff101d-1af2-41c9-8509-e5c68ed4fc29 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.021014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 265/544] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:37.025448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.026282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.026991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.031549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645d7b7f-13e9-4040-ae1f-c7bfab399b2e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:37.047315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:37.049655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:37.052052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:37.075110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:37.075988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.076392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.076787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.088156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:37.094426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:37.095495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:37.096978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:37.099777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 280/545] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:37.107830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.180177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 266/546] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:37.245930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521eb2bb-c060-4479-b3b0-2b70d6fef747 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:37.300418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:37.301351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.301855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.302377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.303221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:37.303707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:37.305261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:37.313637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 281/547] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:37.323296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.415440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:37.451816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b253f2fc-d24d-4346-8e44-0d33f876bde4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:37.479858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.502530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:37.502530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.502530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.502530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.507866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:37.511331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:37.537262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:37.538190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:37.539050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:37.539050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:37.539050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:37.543330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 267/548] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:37.551133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.560270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:37.664485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b17f1d-b7ae-4bc0-b5c8-093522f46f61 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:37.731367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:37.731912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.732184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.732448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.745481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:37.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:37.752739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:37.753698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 282/549] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:37.759129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.769043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462e492f-3f0c-460d-a478-9c35245e16a0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:37.818109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:37.840809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:37.841820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:37.842328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:37.842939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:37.845258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:37.846569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:37.848189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:37.872537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 268/550] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:37.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:37.959711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:37.967596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b33523-070f-4814-b92a-a5c8801e15c8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.030019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.030998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.031473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.031953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.032810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:38.034873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.037600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.056141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 283/551] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:38.059418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:38.066521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb11eb4f-e26e-4dd2-ac0b-6701744d7a3a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.102950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.103944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.104418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.104906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.113744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.115536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:38.129951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:38.131034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:38.131930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:38.134662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 269/552] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:38.146234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:38.234584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:38.265686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15d8d6-c781-42f1-94d6-0936c8c46965 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.308376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52121353-bf45-46ed-9986-730cbbb8d7d9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.312873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.313508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.313788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.314062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.329614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.332398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.348966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.349562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.349836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.350104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.350859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:38.351160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-31d4d3be-7f07-471f-beb8-e514258752dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:38.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:38.352760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:38.353601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:38.353601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:38.353601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:38.353760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00e4f4-a2e9-45c3-bdd0-2f52468bd84c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.358227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 284/553] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:38.358227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce42e-b784-4a5a-8177-80ea930e4d43 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.358227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce42e-b784-4a5a-8177-80ea930e4d43 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.358227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ce42e-b784-4a5a-8177-80ea930e4d43 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.358538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 285/554] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:38.359395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ce42e-b784-4a5a-8177-80ea930e4d43 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:38.382391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 270/555] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Oct 4 10:07:38 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:38.383780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.384066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.384339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.389047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:38.406338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:38.407644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:38.485116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ce42e-b784-4a5a-8177-80ea930e4d43 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.494693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 271/556] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:38.495677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:38.537942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:38.551844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.552675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.553055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.553434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.554181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:38.554935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.557500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.585983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 286/557] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:38.601374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:38.624171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c1808-020a-4ff7-bd51-b8d36e4ce846 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.662573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33affa1b-59a5-49bb-bb34-bf554fcc16d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.663309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33affa1b-59a5-49bb-bb34-bf554fcc16d2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.663677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33affa1b-59a5-49bb-bb34-bf554fcc16d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.664010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33affa1b-59a5-49bb-bb34-bf554fcc16d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.714373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:38.753716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33affa1b-59a5-49bb-bb34-bf554fcc16d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.755452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33affa1b-59a5-49bb-bb34-bf554fcc16d2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.755624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc98087-3d76-41a4-a061-a723acec4396 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.796533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:38.800621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.801989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.801989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.801989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.809506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.810983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4f3ad89b-641b-4eba-bb2e-68cb7ec38cc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:38.822770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:38.823913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:38.824838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 287/558] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:07:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:38.825421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebc775-fe65-4975-a428-a56b8f17a6bc None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.825625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebc775-fe65-4975-a428-a56b8f17a6bc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.825895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ebc775-fe65-4975-a428-a56b8f17a6bc None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.830591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 288/559] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:38 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:38.830591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7e19a-47cf-40ee-bca4-827ccd50ae06 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.830591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7e19a-47cf-40ee-bca4-827ccd50ae06 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.830591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc7e19a-47cf-40ee-bca4-827ccd50ae06 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.831244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 289/560] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:38.834355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7e19a-47cf-40ee-bca4-827ccd50ae06 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:38.891034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 272/561] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:38.891034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccd04d-6e73-40e8-aaa7-a8abf1e718e5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.891034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccd04d-6e73-40e8-aaa7-a8abf1e718e5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.891034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ccd04d-6e73-40e8-aaa7-a8abf1e718e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.900609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-67ccd04d-6e73-40e8-aaa7-a8abf1e718e5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:38.901800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-67ccd04d-6e73-40e8-aaa7-a8abf1e718e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:38.951117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc7e19a-47cf-40ee-bca4-827ccd50ae06 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:38.972909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 273/562] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:38.974147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.974461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.974738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.990323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:38.992423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:38.993794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:38.996041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:38.996636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:38.996906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:38.997173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:38.997897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:38.998195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:38.999631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.019246 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 290/563] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:07:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:39.020057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.062416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.086237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 274/564] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:39.086237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.135097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2472c72-8e8f-44e1-baaf-f4bc24c1326c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.176115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.176775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.177117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.177458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.188470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:39.193949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:39.194888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:39.195783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 291/565] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:39.196450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.210257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba1ab4b-d876-4a8b-b4ec-28ef464012c6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.250806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.257072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.257699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.257974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.258262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.268926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.270405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:39.287468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:39.288563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:39.289669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:39.289669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:39.289669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:39.295838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 275/566] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:39.295838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.308606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d319ed-33f5-4d84-a27b-1e0c9c3a7f17 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.344416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.345141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.345416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.345749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.346616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.356194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.357830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:39.373192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:39.374359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:39.375390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:39.375390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:39.375390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:39.375390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 292/567] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:39.375390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.407148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863056d7-0084-454e-b985-bc41242ad1b2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.440754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.441283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.441559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.441833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.442547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:39.442847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.444210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.452101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 276/568] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:39.453057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.474682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26796665-2134-464c-b8f6-4bde3db25b0c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.476200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.509728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.510381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.510657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.510923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.521188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:39.526470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:39.527587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:39.528440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 293/569] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:39.529107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.554411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e7cf53-8335-4dca-a6ab-96987cc87cab None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.561140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.588430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.589397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.589547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.589845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.590514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:39.590811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.593166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.604242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 277/570] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:39.605024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.636357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64838b2-a97f-4603-becc-22dd0d130e28 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.668722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.669349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.669621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.669891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.680284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:39.685428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:39.686582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 294/571] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:39.687616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.708086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23b34f-fb20-4be7-b7fb-58f1fbfedf0c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.711342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.743115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.743732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.744015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.744291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.744931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:39.745231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.746947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.759567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 278/572] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:39.760352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.788300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.791154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652e5922-4a56-46e7-91bf-86f452bec68a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.824059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.824672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.825056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.825327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.835372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.836819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:39.851810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:39.852938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:39.853993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:39.853993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:39.853993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:39.853993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 295/573] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:39.853993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.865600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7a0a36-73c3-4422-a5c6-f816e90fb19a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.900329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.900922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.901194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.901461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.902130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:39.902462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.903808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:39.911575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 279/574] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:39.912370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:39.932234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.957714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d13ec-f773-473d-8516-05cc65f81482 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:39.989692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:39.990264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:39.990548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:39.991068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:39.997155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:39.997277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:39.998948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.009417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859a628f-1414-4f4c-8b9b-6e54eb811acb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:40.010279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:40.011161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 296/575] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:40.012416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.045000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.045599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.045875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.046141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.046815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.047112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.048453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.056460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 280/576] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.057548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.111749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2123a72f-2b21-4e61-a59d-9522b53e3ff0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.146260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.146879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.147156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.147426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.149275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.154912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.156404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.158592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103905e-3ac5-41ca-8713-a17f2411c7a9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:40.169292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:40.170405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 297/577] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:40.171307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.197884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.198525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.198802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.199076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.199709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.200005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.201361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.212813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 281/578] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.213602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.229387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.276029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf32c86-4934-486f-a6d8-6bd6c350f473 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.308527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.309152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.309429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.309704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.318318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909530da-6125-4b91-a0e9-4406fe9cb2e5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.320916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.322394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:40.336763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:40.337691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:40.338591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:40.338591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:40.338591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:40.338591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 298/579] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:40.339086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.355147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.355750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.356026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.356297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.356981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.357285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.358729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.370398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 282/580] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.371182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.385836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.438773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ab49e3-a527-45aa-bdca-9aba2699003b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.441168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.472918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.473903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.474173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edd10b8-f5ee-4430-9c36-2e19af8d5075 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.474405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.474878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.491113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.492574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:40.506585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:40.507637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:40.509387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:40.509387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:40.509387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:40.509728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.509728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.509728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.510008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 299/581] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:40.510077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.510077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.510238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.510313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.511407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.529878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 283/582] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.531527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.613779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.617402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb81f6cd-0bbe-4b31-bea3-cafd23de3925 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.638637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ac7bd6-5b33-49c5-89ad-8adb1ccb8a87 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.651234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.651856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.652131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.652400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.660207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.661642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.668587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:40.672523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:40.673503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:40.675072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 300/583] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/users/neutron => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:40.675665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.680305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.680918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.681194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.681460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.682096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.682426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.683789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.695138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 284/584] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.695922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.774439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e030fa55-cf41-44c3-bf71-06304cc990b5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.792357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a894d7-7e09-466e-b9dd-9c9972aa76bb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.808560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.809157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.809434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.809706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.810471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=bf8cf97352f244d4b87c5927ee451dd9) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.810778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.812206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.818114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 301/585] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/role_assignments?user.id=bf8cf97352f244d4b87c5927ee451dd9&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.818893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.825858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:40.826518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:40.826796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:40.827067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:40.828357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:40.828675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:40.830538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:40.849068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 285/586] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:40.849068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:40.851943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.912541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:40.961983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3fa5a2-ed7e-44dd-a406-2c5b7d125787 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:40.972815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b73677a-f842-4588-b525-676332936cb1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.001028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.001919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.002193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.002820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/a8514e96615c4785aefde8261120afe3/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.011987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.013008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.013379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.013617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.014303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:41.014614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.016203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.018932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=a8514e96615c4785aefde8261120afe3, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.020373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.032872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 302/587] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:07:40 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/a8514e96615c4785aefde8261120afe3/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:41.036334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 286/588] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Oct 4 10:07:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:41.037523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.040752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.148131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:41.181111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8b3f1c-2448-4e50-a4cc-f9fcd66827ee None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.194543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba247476-1a45-483e-b423-b480c1dca1ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.198655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:41.236748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.238168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.241851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.241851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.247000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.247629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.248899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.248899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.249982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.251488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.258931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.260625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:41.269285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:41.270309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 287/589] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:41.271251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:41.277043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:41.278329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:41.279899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 303/590] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:41.283032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.393296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce418d2-a3a2-428f-9e92-bf5ac602990d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.417841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dc6caa-6a0e-4bf1-a6ab-c31d45940fe2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.442318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.443389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.443675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.444329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.449591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:41.456835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.457536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.458247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.458247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.459156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:41.459570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.460727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.461154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.462580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.480641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 304/591] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:41.481405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:41.482533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:41.483591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:41.483591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:41.483591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:41.483793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.485289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 288/592] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:41.486461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.518910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:41.597105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecff049-1020-4b95-85a2-28ca0e894e84 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.617096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b245c4-072d-411d-9342-0abf50068e80 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.651073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.652159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.652437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.652837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.653720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:41.654024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.655771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.657296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.658150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.658620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.659134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.676053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.681272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 305/593] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:41.681272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.682575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:41.698634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:41.699598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:41.700471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:41.700471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:41.700471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:41.702634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 289/594] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:41.703580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.745704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:41.792676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:41.814890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad03cf-915c-4450-8f7e-38ccf74d0567 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.831203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74695e43-7fd1-4085-9205-702f0175d06b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.852594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.853192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.853469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.853736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.854423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:41.854841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.856053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.872614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 306/595] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:41.872614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.880624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:41.881526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:41.882001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:41.882516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:41.897387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:41.899359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:41.917890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:41.918895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:41.920340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:41.920340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:41.920340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:41.922281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 290/596] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:41.923266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:41.983450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a917dda-63c2-4b7c-92a5-9426ba5ceeba None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:41.992908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.021541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.022166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.022469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.022738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.023377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:42.023813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.025112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.037447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 307/597] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:42.041961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.059440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff9e324-8575-4d03-93ea-09ad6836b743 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.091255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.106661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.107762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.108307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.108728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.117653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.119297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:42.133722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:42.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:42.136011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:42.142594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 291/598] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:42.144361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.153840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6623cb1b-045b-409d-a1da-262951018b64 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.190347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.190948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.191221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.191489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.192121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:42.192416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.193811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.209968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 308/599] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:42.210922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.241067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.271380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d50770a-7a31-4f36-9f0b-c23c6d7ab3e6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.311297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.312873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.313623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.314123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.315848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ecd837-fa48-46f2-b8a7-b7471915a115 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.322556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.324389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:42.337741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:42.338930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:42.339804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:42.347628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 292/600] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:42.347628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.364688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.365631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.366259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.366834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.367777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:42.368279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.370169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.391042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 309/601] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:42.392312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.402054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.465883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a993e054-35d5-4dcd-abdb-728e22b9ffe6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.503480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.504076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.504350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.504619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.511952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.513234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.520238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:42.525554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:42.526848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:42.527808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:42.531409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 293/602] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:42.533200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.536787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dd901b-4b93-4e57-8d7a-11095dd18eaf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.582626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.583479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.583783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.584495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.585472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:42.585783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.587879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.606301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 310/603] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:42.607169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.653243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74f0f6-e80f-4a3e-be29-04f32f8e1240 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.656825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.690756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.691702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.692121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.692576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/0e568115c67243d3904359b84e0a790e/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.708812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=0e568115c67243d3904359b84e0a790e, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.710495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.734423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 294/604] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:07:42 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/0e568115c67243d3904359b84e0a790e/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:42.736398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.749291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474e7612-36b9-4019-8fc7-4c4a938b2fde None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:42.821263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:42.821808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:42.822082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:42.822381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:42.823094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:42.823398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:42.824857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:42.859121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 311/605] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:42.869160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:42.894640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:42.984786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07994b-42b4-46c4-9ecf-a3f69de31d62 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:43.011128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5b9c4-cd85-4d4b-9737-fa629ecaa334 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:43.021628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44b13b-cb4f-4aea-8c00-4c7d5008049c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:43.022314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44b13b-cb4f-4aea-8c00-4c7d5008049c None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.022596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44b13b-cb4f-4aea-8c00-4c7d5008049c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.022880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44b13b-cb4f-4aea-8c00-4c7d5008049c None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/99de0be14e684b3696a4570987119ea7/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.038697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44b13b-cb4f-4aea-8c00-4c7d5008049c None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=99de0be14e684b3696a4570987119ea7, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:43.040142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44b13b-cb4f-4aea-8c00-4c7d5008049c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:43.052339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e52296-ebee-4fda-a622-9bdba6111300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:43.054150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e52296-ebee-4fda-a622-9bdba6111300 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.054664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e52296-ebee-4fda-a622-9bdba6111300 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.055162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e52296-ebee-4fda-a622-9bdba6111300 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/d5c6c66598c74c11a1c9cac119644c19/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.062231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 295/606] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:07:42 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/99de0be14e684b3696a4570987119ea7/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:43.087994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e52296-ebee-4fda-a622-9bdba6111300 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=d5c6c66598c74c11a1c9cac119644c19, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:43.089660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e52296-ebee-4fda-a622-9bdba6111300 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:43.102983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940bd19-16ec-4ed2-b7fa-7dd98d7d3c6a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.103318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940bd19-16ec-4ed2-b7fa-7dd98d7d3c6a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.103601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4940bd19-16ec-4ed2-b7fa-7dd98d7d3c6a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.105511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 296/607] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:07:43.116820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 312/608] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:07:42 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/d5c6c66598c74c11a1c9cac119644c19/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:43.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcf4e4-af88-4dc6-8bd8-fe436759cd76 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.120862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcf4e4-af88-4dc6-8bd8-fe436759cd76 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.121145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dcf4e4-af88-4dc6-8bd8-fe436759cd76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.137968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:43.143295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-26dcf4e4-af88-4dc6-8bd8-fe436759cd76 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:43.144578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-26dcf4e4-af88-4dc6-8bd8-fe436759cd76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:43.225941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 297/609] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:43.450841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3ba58-992a-429e-9d48-dbe5a9bcc239 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.451209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3ba58-992a-429e-9d48-dbe5a9bcc239 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.451389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f3ba58-992a-429e-9d48-dbe5a9bcc239 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.452996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 313/610] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:43.459115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.459676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.459676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.484094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:43.485272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:43.525749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3ba58-992a-429e-9d48-dbe5a9bcc239 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:43.555881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 298/611] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:43.632839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:43.687968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f3ba58-992a-429e-9d48-dbe5a9bcc239 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:43.732821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:43.733697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.734124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.734558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.754563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:43.759686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:43.760829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:43.761880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:43.763874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 314/612] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:43.769839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:43.859190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:43.912287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd51be4-5fb0-4028-bc5f-fb7da93aae92 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:43.960746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:43.961358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:43.961636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:43.961905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:43.962584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:43.962932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:43.964582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:43.979306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:43.981084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 299/613] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:43.987316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.090239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e831a74-45af-4879-b9dc-8221646a5a3b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.116930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:44.125633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.126280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.126555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.126926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.137973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:44.140468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:44.165378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:44.166489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:44.168424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:44.168424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:44.168424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:44.168424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 300/614] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:44.177851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.231362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be12f841-e24c-4b20-9a4c-7416fd4b85e2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.258533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:44.277038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.277657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.277927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.278191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.299533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:44.307325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd53bc3e-44c5-49ee-b701-fe6aea5b8ef2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:44.310583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:44.311535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:44.312368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:44.318470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 315/615] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:44.319457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.375289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.375893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.376167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.376441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.377666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:44.380481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:44.382671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:44.411399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 301/616] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:44.417914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.491932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:44.525086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3eeaf-862b-456a-9a3d-e29f166ed2b0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.559997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb7c94-05bd-45ed-9dc7-6f15f15bac19 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.598000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.598675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.598949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.599219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.606676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:44.608052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:44.610898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.611807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.612093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.612371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.613050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:44.613353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:44.614780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:44.628177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 316/617] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:44.628768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:44.629710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:44.630677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 302/618] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:44.638511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.639762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.722726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:44.810731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:44.815945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fba3c4c-df72-4f41-a404-067a882519d5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.821359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be36a77-0d73-418d-a21e-fce22606e52e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:44.879267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.879874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.880151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.880421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.881443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:44.881748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:44.883206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:44.894438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:44.895179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:44.895455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:44.895725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:44.913284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 317/619] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:44.914038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:44.915496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:44.918097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:44.941388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:44.942631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:44.943708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:44.943708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:44.943708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:44.943708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 303/620] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:44.949671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.087324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:45.092149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c927f-e95d-4922-8ab9-517d723390e9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.125458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb31b5cd-cd60-47c3-93fb-1e5132b379d2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.160605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.161232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.161503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.161769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.175638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.176249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.176524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.176793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.177486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:45.177782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.179162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.182388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.185013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.203758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 304/621] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:45.208842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:45.209770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:45.210692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:45.210692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:45.210692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:45.214844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 318/622] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:45.216416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.220370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.235156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:45.387328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc28fb6-e765-4241-a4ab-7a0aac4017a2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.400084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633810fd-7663-44dc-ad71-5f5901afd6a5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.431089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.432209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.433052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.433744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.437248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.437850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.438119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.438429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.439072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:45.439505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.441092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c9c73-76f4-4e36-b0b7-bd852bcd1a75 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.445587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.447345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.456610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 319/623] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:45.458711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a943cd-3e1b-49ba-8a1e-3b62619b9838 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.458994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a943cd-3e1b-49ba-8a1e-3b62619b9838 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.459262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a943cd-3e1b-49ba-8a1e-3b62619b9838 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.464933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 320/624] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:45.464933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a943cd-3e1b-49ba-8a1e-3b62619b9838 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-90b7b7e1-8e38-4afa-82a9-06d1ac88d18f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:45.467874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:45.469015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:45.470232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:45.473473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 305/625] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:45.474836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.475260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.475678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.485699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:45.502172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:45.503702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:45.577085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:45.590238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 306/626] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:45.590238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.649498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a943cd-3e1b-49ba-8a1e-3b62619b9838 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.723179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.724317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.724802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.725216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.737743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.740373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-6903a4d1-7747-46f7-af69-a8376881b09a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:45.762234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:45.763371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:45.764442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:45.766566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 321/627] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:45.768139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079957f7-64cc-4677-b9fa-532e9405feed None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.768599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079957f7-64cc-4677-b9fa-532e9405feed None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.769054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079957f7-64cc-4677-b9fa-532e9405feed None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.771028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 322/628] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:45.772018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079957f7-64cc-4677-b9fa-532e9405feed None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.776785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4796d2f0-1510-4e33-b824-fea64e9514c6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.817116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.817726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.818003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.818654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.819383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:45.820005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.821422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5724b1d4-5cfe-42ae-a137-7563cbc9e889 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.844356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 307/629] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:45.844356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.844356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.844356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.846444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:45.861727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:45.862947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:45.880284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079957f7-64cc-4677-b9fa-532e9405feed None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:45.916767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:45.917285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.917563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.917836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.918574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:45.918882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:45.919942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:45.920314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d976a-66a8-4ec4-9317-0691c522c8af None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:45.932194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 323/630] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:45.933439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5a9e1-214e-4383-afaa-4eb3d5bf64e3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:45.933727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5a9e1-214e-4383-afaa-4eb3d5bf64e3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:45.934009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5a9e1-214e-4383-afaa-4eb3d5bf64e3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:45.935179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 308/631] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:45.935676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 324/632] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:45.936293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:45.936450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5a9e1-214e-4383-afaa-4eb3d5bf64e3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.039808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b6ce7-b3cb-4d5a-b9c6-efa9e7503ac9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.049086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5a9e1-214e-4383-afaa-4eb3d5bf64e3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.075605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2fd94b-dbdb-4c4e-9e69-9fab3765d5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.076211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fd94b-dbdb-4c4e-9e69-9fab3765d5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.076484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fd94b-dbdb-4c4e-9e69-9fab3765d5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.076754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2fd94b-dbdb-4c4e-9e69-9fab3765d5b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.081284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.088970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.089723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.089995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.090303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.100941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:46.105665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:46.106650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 325/633] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:46.107501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.138921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.159646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fd94b-dbdb-4c4e-9e69-9fab3765d5b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:46.161310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2fd94b-dbdb-4c4e-9e69-9fab3765d5b2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:46.216441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec6767a-4275-4fe2-825f-62c6511d3b9e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.249888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.250563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.250838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.251104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/bf8cf97352f244d4b87c5927ee451dd9/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.261494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 309/634] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:46.262667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.262948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.263227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.266275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=bf8cf97352f244d4b87c5927ee451dd9, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:46.268311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:46.281683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 326/635] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:07:46 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/bf8cf97352f244d4b87c5927ee451dd9/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:07:46.282369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:46.282487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.284191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:46.328840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.368571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 310/636] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:46.369454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.391439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.400652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c1e8ca-77f6-468e-840e-77626a20b0a5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.442250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.443261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.443890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.444454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.445333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:46.445848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:46.447511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:46.458169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 327/637] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:46.459428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.489270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f413394d-a0bc-4959-bbd1-e37fc39b8e3b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.537122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.537790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.538037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.538328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.553314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:46.558377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:46.559367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:46.560290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 311/638] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:46.561019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.586396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6386c04e-265a-437a-98d6-9a81710138d7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.588391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.636056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.636889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.637323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.637748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.649432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:46.651173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:46.656485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:46.666779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:46.667951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:46.669090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:46.669090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:46.669090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:46.670549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 328/639] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:46.671624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.683295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dac43d7-3335-4023-8eeb-b095767594cf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.718824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.719514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.719791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.720061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.731441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:46.733242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:46.749004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:46.749953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:46.758801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:46.758801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:46.758801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:46.758801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 312/640] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:46.758801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.787511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399729a-f47b-4cd8-81ff-0a3eacd74e85 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.837606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.838323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.838864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.839145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.839818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:46.840122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:46.841587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:46.850068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 329/641] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:46.850977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.873391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.887528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388a0a2-c42c-42c1-8f61-0f0f14ef862d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.917215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:46.923331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:46.924920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:46.925131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:46.925555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:46.942838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:46.948167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:46.949132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:46.949969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:46.954309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 313/642] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:46.955962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:46.964170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08619c-865e-4dea-b092-52d16512ed15 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:46.999147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.000008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.000284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.000551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.001194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:47.001493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.003059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.015631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 330/643] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:47.016761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.070075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc8935-861d-4631-adb5-65a0a1310102 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.096611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.116314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.117527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.117816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.118382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.119029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:47.119335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.120035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11992b0-eaa7-4721-8440-7b0150762df2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.120722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.133216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 314/644] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:47.134396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.166240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.166870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.167146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.167409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.180429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.182701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.194762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:47.198863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:47.199822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:47.200862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:47.200862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:47.200862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:47.200862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 331/645] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:47.200862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.260043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72be8e4-566f-4849-85b7-686886270672 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.297430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.298539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.299115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.299564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.300357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:47.300946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.302601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.306970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501eb006-0b05-4189-8570-9a9450980d79 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.311501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 315/646] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:47.312486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.337251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.347452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.348079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.348352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.348620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.355274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.356720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:47.368873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:47.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:47.370693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 332/647] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:47.371678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.425536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3cabee-9d02-4293-a9ca-2d6c8f2a7338 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.439062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.462048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.462791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.463064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.463331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.471651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3c74eb-7ecf-45c1-a44a-0c0df6c5b9c0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.472916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.474708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:47.486269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:47.487222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:47.488188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 316/648] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:47.488802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.510012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.510708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.510933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.511201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.511839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:47.512137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.513519 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.525380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 333/649] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:47.526171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.585719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.597323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c7837-47fb-4710-a19e-a586cb6d83c8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.644179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.644812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.645092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.645364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.646912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2bdc80-43bc-477c-8b62-6964758cffc2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.657102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.659179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:47.674415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:47.675530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:47.676523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:47.676523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:47.676523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:47.676523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 317/650] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:47.680819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.687718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.688321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.688595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.688860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.690190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:47.690520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.691949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.708113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.708281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 334/651] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:47.709299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.789734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a3183-79ab-475a-933c-c9084ea22e74 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.811230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b624aebf-d3ca-47ef-813b-b060785d74d7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.825707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.833896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.834659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.835120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.835586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.836498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:47.836958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.838889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.855806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:47.856428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:47.861877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:47.861995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 318/652] 10.222.0.13 () {56 vars in 1139 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:47.862090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] PATH_INFO: `/v3/users/glance` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:47.862176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.869753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:47.871260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:47.882805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:47.883831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 335/653] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:47.884772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:47.955752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:47.976575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bdc024-2d77-41f1-9e59-34e50de7060b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:47.984522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e472a2a-96cd-44b4-aeaf-b5585f7c55da None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.014587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.015196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.015470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.015741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.016372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:48.016694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.018037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.019720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.020329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.020625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.020869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.021597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=a8514e96615c4785aefde8261120afe3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:48.021905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.023362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.029329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 336/654] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/role_assignments?user.id=a8514e96615c4785aefde8261120afe3&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:48.030110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.041642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 319/655] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:48.042802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.076231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.156288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88277b7c-62c0-49c8-ac74-d1e8c32706a3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.168619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d220c940-5e61-435d-9a11-3aa97ae939e0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.197056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.197866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.198283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.198658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.211278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.212185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.212605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.213028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.214309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:48.214754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.216344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.233352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.239348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 320/656] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:48.242411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.284630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.286844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.321961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.355402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f4ecac-d654-4bc1-9048-e8cbc0c2b912 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.395003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.395914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.396336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.396761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.404947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.406881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.414279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 337/657] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:48.415197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:48.421573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:48.422757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:07:48.424454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:48.426444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 321/658] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:48.427483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.499143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.543094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58381b8-d963-4b02-af12-c9257797a7fb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.549653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb33c270-c7b1-46e0-bc83-3c69e8b7aa43 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.578162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.584775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11de5255-0bd6-42c0-bcb2-4dcd6a900e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.585407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de5255-0bd6-42c0-bcb2-4dcd6a900e8f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.585680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de5255-0bd6-42c0-bcb2-4dcd6a900e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.585776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.585980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11de5255-0bd6-42c0-bcb2-4dcd6a900e8f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.586395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.586674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.586991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.598236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.599872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:48.614459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:48.615421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:48.621344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:48.621344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:48.621344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:48.621344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 322/659] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:48.621344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.671070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de5255-0bd6-42c0-bcb2-4dcd6a900e8f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.672547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11de5255-0bd6-42c0-bcb2-4dcd6a900e8f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.729027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d8fb61-3329-43c6-a1dc-58bec99b58c8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.744667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.773052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.773644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 338/660] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:48.774754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21939de-3a64-4c02-8a0c-435aedeebe73 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.775038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21939de-3a64-4c02-8a0c-435aedeebe73 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.775308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21939de-3a64-4c02-8a0c-435aedeebe73 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.778236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 339/661] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:48.778236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21939de-3a64-4c02-8a0c-435aedeebe73 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.778453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.778453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.778793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.779400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:48.779704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.781138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.805433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 323/662] 10.222.0.13 () {56 vars in 1137 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:48.805433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:48.815424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:48.901190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21939de-3a64-4c02-8a0c-435aedeebe73 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.914845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8953d50a-5a2d-4169-a4b6-6b3ed21c6cd5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:48.936861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.937859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.938141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.938464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.939732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:48.940048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.941532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77105781-bfc3-477c-b9df-f44da6e2db70 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.952789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:48.953420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.953876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.953981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 340/663] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:48.955677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:48.956056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:48.956364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.959182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:48.970768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:48.972198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:48.974533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:48.975726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:48.987130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:48.988258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:48.989300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:48.989300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:48.989300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:48.989300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 324/664] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:48.989300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.014368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.054243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 341/665] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:49.055115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.092251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.117997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e4eb12-bd79-41da-ba36-340b411fcf93 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.157203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.157775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.158375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.158375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.188222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50b814f-bc7d-4f1e-9575-5239c442d799 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.237121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.237613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.238381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.238926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.239107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.239584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.253626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.255137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.266003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:49.266918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:49.268476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:49.269877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 342/666] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:49.270727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.359179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.361794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 325/667] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:49.362947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.381783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8ac178-0654-4cb0-ac7a-77b7d1f4547c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.417473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.418101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.418473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.418825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.419543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:49.419917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.421361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.438181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 343/668] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:49.438181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.476449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cf2be0-801c-4b7d-a83a-2d66e9a33ae7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.503868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.516796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.518223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.518641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.519036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.519879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:49.520285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.521811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.540668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 326/669] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:49.540668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.545581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b37fad-696e-4b57-8205-4dae260a9193 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.582864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.583572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.583913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.584245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.599544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:49.604538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:49.605423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:49.606715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 344/670] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:49.607415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.609855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.653287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabec50d-9f1c-4b6f-8319-3406df5b2d0a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.691062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.691780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.692172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.692547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.703889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.706705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.716009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d35f2b-f7e4-4b90-a9c2-4b689f2882d7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:49.721629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:49.722819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:49.723929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:49.723929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:49.723929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:49.725129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 327/671] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:49.725755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.760239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.760898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.761176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.761452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.763176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.769088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.770582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:49.781757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:49.783347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 345/672] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:49.784679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.840642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ce7565-ab63-4612-b6ae-620701200874 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.850673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:49.879460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.880230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.880613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.880892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.881652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=0e568115c67243d3904359b84e0a790e) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:49.881932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.883417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.889506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 328/673] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/role_assignments?user.id=0e568115c67243d3904359b84e0a790e&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:49.890331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:49.904573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2f0669-c9f1-4d94-ac81-4e7d33a433d3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:49.950774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:49.952333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:49.952898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:49.953882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:49.954858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:49.955355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:49.957810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:49.975003 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 346/674] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:49.975524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.040221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.046118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680aaf5-dca5-4066-95bc-f6166d7579a8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.096353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.096353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.096654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.096727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.097349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:50.097645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.099140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.112416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 329/675] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:50.113250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.119093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3a8af9-7123-4243-a1f0-2b320c536ef6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.145853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.161249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.161863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.162142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.162445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.163143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:50.163449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.165321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.186800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 347/676] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:50.186800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.268295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0670f2a-a199-49ea-a19d-6d7fec8f970d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.310608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.311239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.311512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.311781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.312681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2306230-4c0b-472c-9111-157293c285c6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.324948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.326435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:50.344396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:50.345979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:50.347092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:50.347092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:50.347092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:50.347092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 330/677] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:50.350972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.353608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.354275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.354553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.354822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.362858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.364323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.365821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:50.376695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:50.377797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:50.378721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:50.382369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 348/678] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:50.383517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.432897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.463773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b03c41-0f17-407d-99e7-cd4f0d33e879 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.494813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b94a73-cf8a-4f6d-aa0a-da5c47c8ea51 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.497964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.498594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.498869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.499136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.499878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=d5c6c66598c74c11a1c9cac119644c19) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:50.500181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.501624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633d9af9-75ef-4bf3-b84b-4bcf31954135 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.518233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 331/679] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/role_assignments?user.id=d5c6c66598c74c11a1c9cac119644c19&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:50.518233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebbf74-1fc5-4f6f-b407-91dd5e2c3a4a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.518233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebbf74-1fc5-4f6f-b407-91dd5e2c3a4a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.518233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faebbf74-1fc5-4f6f-b407-91dd5e2c3a4a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.518233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 332/680] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:50.518233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebbf74-1fc5-4f6f-b407-91dd5e2c3a4a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.545977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34e4308-685c-4264-878f-356708493019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.547216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4308-685c-4264-878f-356708493019 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.547500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4308-685c-4264-878f-356708493019 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.547772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e4308-685c-4264-878f-356708493019 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.549460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4308-685c-4264-878f-356708493019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:50.549770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4308-685c-4264-878f-356708493019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.551980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34e4308-685c-4264-878f-356708493019 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.571023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 349/681] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:50.572878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.573166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.573437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.597446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:50.598872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:50.621762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.659502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faebbf74-1fc5-4f6f-b407-91dd5e2c3a4a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.691353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 350/682] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:50.692375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.715196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.715196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.715196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.715196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.715786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:50.716310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.717963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.730593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.741570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 333/683] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:50.751202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.808342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a5dcd0-5e47-4462-8a95-e03dbf30c57a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.843561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.844195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.844479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.844750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.852118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.854192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:50.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:50.866770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:50.867698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 351/684] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:50.870598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d0144-8f69-4d0b-8695-a39121ae5987 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:50.872659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:50.914372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:50.915336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:50.915832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:50.916419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:50.917339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=99de0be14e684b3696a4570987119ea7) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:50.917840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:50.919798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:50.921925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.929488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 334/685] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/role_assignments?user.id=99de0be14e684b3696a4570987119ea7&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:50.983399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:50.990706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd2bc5-cd16-4e2d-a2d2-7fa260a78076 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.010376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.029090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.029988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.030450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.030930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.031829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:51.032284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.034469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.054229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 352/686] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Oct 4 10:07:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:51.060398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.164084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f57cfd-8b49-4ab0-a95a-86a703d3c2d3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.190186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60263300-819b-4281-8c2e-d96aef0c822a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.221911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.222603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.222878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.223148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.241977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:51.244866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.245710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.246190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.246703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:51.247622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:51.248504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:51.249333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:51.255049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 335/687] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:51.255843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.277946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:51.318564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:51.334634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.336567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.362113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c209797-fd29-4db1-99c0-bc0f5f33a48f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.397333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.397944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.398235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.398720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.399174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:51.399474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.400854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.414288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 336/688] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:51.416990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.451472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 353/689] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:51.461492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.529685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89df94d-15af-41ba-876c-365f84d47ee8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.554574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:51.567416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.568055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.568338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.568666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.569948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:51.581363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.583169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:51.600162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:51.601333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:51.602377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:51.602377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:51.602377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:51.608198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 337/690] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:51.609921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.728892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dda5a1-ed42-46a8-8a6d-977b6b12814a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.730398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18698abf-027c-4225-9b0b-18206d5dd3ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.767434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.768481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.768995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.769553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.770399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:51.770878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.772438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.790557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 338/691] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:51.792816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.830728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:51.859028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.859877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.859999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.860230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.895044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.896649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.902495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94098ff-adce-4e54-97b4-6a19cb09a7e8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:51.929545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:51.930656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:51.936085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:51.936085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:51.936085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:51.936085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 354/692] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 470 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:51.939962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:51.955880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:51.956882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:51.957366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:51.958432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:51.967351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:51.969957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:51.984777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:51.986442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:51.987383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:51.994268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 339/693] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:51.995826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:52.105994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deda135-1746-4c54-a768-82bf47fb32e5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:52.146439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:52.147296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:52.147959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.148827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.148827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.149547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:52.149975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:52.152411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:52.177906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 340/694] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:52.179243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:52.260291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d89a5-1b4a-4147-9b40-5d327379b094 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:52.269642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:52.327971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:52.328618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.329151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.330086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.331477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:52.331787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:52.332361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496c4515-3366-43b9-ac2c-75980979d67d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:52.333207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e0f57d-3922-4cfa-b3de-2f93e4018378 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:52.350195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 355/695] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:52.351803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107519-c4b5-4196-bde3-8ec6c805b228 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.352089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107519-c4b5-4196-bde3-8ec6c805b228 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.352365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc107519-c4b5-4196-bde3-8ec6c805b228 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.357702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 356/696] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:52.357702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107519-c4b5-4196-bde3-8ec6c805b228 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:52.377395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743fafa8-a42b-4fc3-8cf6-ca666aa1f0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:52.378281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fafa8-a42b-4fc3-8cf6-ca666aa1f0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.378698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fafa8-a42b-4fc3-8cf6-ca666aa1f0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.379245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743fafa8-a42b-4fc3-8cf6-ca666aa1f0f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.400151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:52.452670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fafa8-a42b-4fc3-8cf6-ca666aa1f0f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:52.454132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743fafa8-a42b-4fc3-8cf6-ca666aa1f0f4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:52.498893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc107519-c4b5-4196-bde3-8ec6c805b228 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:52.540677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:52.541773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.542339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.542845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.551213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:52.552981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:52.560572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 341/697] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:52.561814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.562092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.562403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:52.567709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:52.568616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:52.569516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:52.571963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 357/698] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:52.573448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:52.585466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:52.585466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:52.605932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:52.661943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:52.694506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 342/699] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:52.695437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:52.718549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c40b26-37ad-4b8a-8bdc-d1343a2cd4b9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:52.788901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:52.790441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.792665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.793138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.817313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:52.820221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:52.852390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:52.853362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:52.854249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:52.854249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:52.854249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:52.859075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 358/700] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:52.864699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:52.899844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01a443-9ce1-421e-9f07-bab743f3b295 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:52.958434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:52.959058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:52.959337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:52.959720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:52.960423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:52.960777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:52.962236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:52.975879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 343/701] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:52.980573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.004967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.027208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.036666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d16d2d-2ea8-400f-abff-3babe023ed5f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.078317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.079911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.080397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.080856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.081902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:53.082494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.084095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.091228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246a575b-34c2-4301-ab7f-1f562ee41e5a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.098419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 359/702] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:53.099527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.135827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.136511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.136793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.137075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] PATH_INFO: `/v3/services/network` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.137759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.139320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.219096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7aeac-fd15-4394-a37a-b759a1ccc6c3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.258855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.259856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.260325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.260799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.261746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=bf8cf97352f244d4b87c5927ee451dd9) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:53.262279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.263955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.267718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:53.270399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:53.271529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:07:53.272403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:53.275042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 344/704] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:52 2022] GET /identity/v3/services/network => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:53.276101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.281633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 360/704] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/role_assignments?user.id=bf8cf97352f244d4b87c5927ee451dd9&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:53.291842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.401466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3948cfee-b5f8-4284-bc1b-9d1e25f81932 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.404629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.442941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.443978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.444459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.444982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.453440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.455832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.461629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880479dd-767e-4e58-ba9c-aaaf135c73d3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-6b920b43-bf09-4789-930b-96316ab9d486 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:53.469599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:53.471097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:53.471982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:53.479157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 345/705] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:53.481127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0f91-b02d-47ea-85fa-fdde7e388347 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.481720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0f91-b02d-47ea-85fa-fdde7e388347 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.482222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac0f91-b02d-47ea-85fa-fdde7e388347 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.484789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 346/706] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:53.486006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0f91-b02d-47ea-85fa-fdde7e388347 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.500547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.501047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.501908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.502347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.503058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] RBAC: Adding query filter params (name=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:53.503437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.504754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc97ed-beeb-4c2d-b3e3-6b1ecefc4150 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.514591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 361/707] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/services?name=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:53.514591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.514591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.514591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.531883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:53.534019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:53.602560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.614244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 362/708] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:53.614244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.626464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac0f91-b02d-47ea-85fa-fdde7e388347 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.676996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.677771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.678058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.678371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.679076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:53.679390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.681601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6660f51-d297-4a80-8fed-697e7413000c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.697745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 347/709] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:53.699211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f843a-4867-47e9-b406-8878061432d8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.699560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f843a-4867-47e9-b406-8878061432d8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.699835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f843a-4867-47e9-b406-8878061432d8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.701347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 348/710] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:53.705134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f843a-4867-47e9-b406-8878061432d8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.732743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455b724-ad7a-4906-9d0e-405e4f439091 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.740747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.778115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.778744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.779022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.779291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.779917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] RBAC: Adding query filter params (type=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:53.780219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.781585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486f0e8b-312a-4056-9213-fa1fd14122c0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.788346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 363/711] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/services?type=network => generated 124 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:53.789634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.789921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.790193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.810160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:53.811419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:53.883927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9f843a-4867-47e9-b406-8878061432d8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:53.935282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:53.954373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 364/712] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:53.955570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:53.968148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:53.968854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:53.970928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:53.970928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:53.970928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=a8514e96615c4785aefde8261120afe3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:53.970928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:53.973285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:53.982137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 349/713] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/role_assignments?user.id=a8514e96615c4785aefde8261120afe3&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:54.087946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d3c922-0c4d-435a-ae0f-d449434da483 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:54.126885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:54.127823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:54.128242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:54.128654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:54.140515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:54.142490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:54.160099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:54.161356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:54.162466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:54.162466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:54.162466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:54.164249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 365/714] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:54.170139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:54.300515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:54.321355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5328f35c-c287-48e3-8838-62606c26bac9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:54.346977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:54.358064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:54.358705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:54.358978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:54.359245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:54.359879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:54.360178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:54.362622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:54.375472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 350/715] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:54.382434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:54.399485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:54.516435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994ad198-f648-4094-af3a-6d89e9a73d0f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:54.528130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4af0d2-2c79-4dba-8d1b-78f2de4eb0e6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:54.561179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:54.562404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:54.564810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:54.565520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:54.567041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:54.569609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:54.607661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:54.609052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:54.609532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:54.610169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 351/716] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:07:54 2022] POST /identity/v3/users => generated 254 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:54.611721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:54.612629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:54.620197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:54.640181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:54.641215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:07:54.642708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:54.645823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 366/717] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Oct 4 10:07:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:54.651400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:54.667469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:54.788659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdc0506-a26f-440b-b315-8918896488b9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:54.848166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:54.848801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:54.849080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:54.849356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:54.849997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:54.850326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:54.852743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a5a66-4473-4af7-8e93-5fec4b31b292 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:54.862945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 352/718] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Oct 4 10:07:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:54.868472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:54.984708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62548bb7-5b67-420e-a61d-49f217fdd219 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:54.987537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:55.023355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:55.024327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:55.024829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:55.025331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:55.026188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:55.026740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:55.028328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bbe5e6-03f0-49c1-be55-db3f8626eefd None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:55.034393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 367/719] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Oct 4 10:07:54 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:56.045600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a7b6e-2efa-4689-8c3a-e31bc59f6949 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.046091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a7b6e-2efa-4689-8c3a-e31bc59f6949 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.046091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a7b6e-2efa-4689-8c3a-e31bc59f6949 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.047687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 353/720] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:56.053334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b012-3ba2-492f-897c-f3522401de93 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.053774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b012-3ba2-492f-897c-f3522401de93 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.053852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe4b012-3ba2-492f-897c-f3522401de93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.079221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-efe4b012-3ba2-492f-897c-f3522401de93 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:56.080439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-efe4b012-3ba2-492f-897c-f3522401de93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:56.170855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 368/721] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:56.580520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a217-9673-4714-baac-456f6dd11756 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.581216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a217-9673-4714-baac-456f6dd11756 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.581699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac92a217-9673-4714-baac-456f6dd11756 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.583736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 354/722] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:56.589222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308e748-d96d-4742-86a8-14e368542a75 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.589788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308e748-d96d-4742-86a8-14e368542a75 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.590324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a308e748-d96d-4742-86a8-14e368542a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.620601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a308e748-d96d-4742-86a8-14e368542a75 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:56.622827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a308e748-d96d-4742-86a8-14e368542a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:56.653647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92a217-9673-4714-baac-456f6dd11756 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:56.727604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:56.730574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 369/723] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:56.738227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde0e9-4f01-4773-8252-ffa52a25f7cd None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.738789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde0e9-4f01-4773-8252-ffa52a25f7cd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.739264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde0e9-4f01-4773-8252-ffa52a25f7cd None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.740801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 370/724] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:56 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:56.746550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.747041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.747502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.770479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:56.771992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:56.775705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac92a217-9673-4714-baac-456f6dd11756 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:56.797797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:56.839774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:56.840334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:56.840607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:56.840877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:56.851436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 371/725] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 04 10:07:56.866865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:56.871604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:56.872500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:56.873324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 355/726] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:56.878825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.034272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0a94ab-8cfe-4aad-8f02-887dac8074a4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.068549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.069821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.070260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.070666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.071475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:57.071916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.073516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.082632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 372/727] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:57.090320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.116706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.154553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:57.248259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5251c6b-88b0-424b-a339-69a77dfc8458 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.252128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b88e93-d626-4f23-9e4a-8ecffe516014 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.280810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:57.289568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.290310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.290589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.290857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.291617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.292412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.292973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.293465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.293953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.294442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.306581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.308709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.320015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 373/728] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:07:57 2022] POST /identity/v3/services => generated 240 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:07:57.320015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:57.335292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:57.336291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:57.337184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:57.337184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:57.337184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:57.337184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 356/729] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:57.347413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.446589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:57.494065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950adb19-2853-4572-bab9-4ee609a7f389 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.546462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.547100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.547377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.547647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.552275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869c240-365b-4e49-bbb1-71e711d8e1fc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.560883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:57.566744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:57.567710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:57.568572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 374/730] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:57.573894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.613613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.615170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.615693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.616175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.617410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:57.617938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.619833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.638254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 357/731] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:57.642735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.691572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:57.731536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40906fe7-556a-4925-b5ad-8c31d0a1c1ce None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.765566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bf688a-c0fe-4c52-af58-93abf9d9149b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.774918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.775724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.775999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.776621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.777293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:57.777887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.779663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3592885-ccc8-48fc-9acf-48a8c099cf0d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.787242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:57.792915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 375/732] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:07:57.794179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26fb44-3625-4efc-a82e-44e934bf4bc3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.794452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26fb44-3625-4efc-a82e-44e934bf4bc3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.794725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da26fb44-3625-4efc-a82e-44e934bf4bc3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.796066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 376/733] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:57.800123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26fb44-3625-4efc-a82e-44e934bf4bc3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.810232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.810881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.811160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.811437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.821124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.822620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4b88dbe7-5f0e-4351-b58f-b4a2cd2889ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:57.836628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:57.837632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:07:57.839164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 358/734] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:57.843375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14166259-19db-49db-891c-979161648d68 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.843650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14166259-19db-49db-891c-979161648d68 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.843921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14166259-19db-49db-891c-979161648d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.863164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-14166259-19db-49db-891c-979161648d68 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:57.864395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-14166259-19db-49db-891c-979161648d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:57.923106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da26fb44-3625-4efc-a82e-44e934bf4bc3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:57.933093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 359/735] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:57 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:57.934154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebeb97-6614-41cf-9905-f588526e7186 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.934476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebeb97-6614-41cf-9905-f588526e7186 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.934737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ebeb97-6614-41cf-9905-f588526e7186 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.940570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 360/736] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:57.940570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebeb97-6614-41cf-9905-f588526e7186 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:57.969197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:57.970013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:57.970424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:57.970787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:57.975144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:57.981183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:57.982785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1de1ecba-c3f0-462e-acbd-725311ee147a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:57.998667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:57.999630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:58.000579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:58.000579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:58.000579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:58.005178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 377/737] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:07:58.006694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.007036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.007368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.026888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:07:58.028162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:07:58.056269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ebeb97-6614-41cf-9905-f588526e7186 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.082603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:58.093601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.094466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.095294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.095688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.096644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:58.097258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.099026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.107804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 378/738] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:07:58 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 04 10:07:58.108893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.117654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 361/739] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Oct 4 10:07:57 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:58.129301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.255407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1601b1d-56d3-4093-8945-bdbbf95f9a5e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.266621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:58.282623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cd8a04-d1b0-4130-bb57-90b41d369470 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.294390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.295131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.295501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.295896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.296655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:58.297106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.299227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.311681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 379/740] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:58.312730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.320602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.321212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.321494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.321769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.358562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:58.396091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.397475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.435249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0529065e-55b7-421a-9ed7-1da037a64943 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.472716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.473358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.473638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.473908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.484109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:07:58.489090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:07:58.492695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:58.493828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:07:58.494728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:58.495354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 380/741] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:58.496187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.516487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 362/742] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 04 10:07:58.518315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.603517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:58.607284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897d5aae-bb9d-4dbc-9ed8-b7161dff4739 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.623300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6b4aa9-c33a-4aa5-a798-04388fa4c383 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.647805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.648456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.648826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.649120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] PATH_INFO: `/v3/users/nova` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.656191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.658254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.660240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.660872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.661142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.661413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.662053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.663441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.681543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 363/743] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:07:58 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:07:58.682524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.755601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:58.789711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:58.791039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:07:58.791950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:58.798594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 381/744] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:58.800004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.837386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96b7b3-6d33-4674-b6d6-657f70ce2b3d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.881964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.882901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.883187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.883458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.884283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:58.884579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.886792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.897694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 364/745] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:58.899119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:58.917904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9f9576-0e3c-45a0-a6b0-c6c1ec8459ca None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:58.960225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:58.960854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:58.961128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:58.961397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:58.973205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:58.974782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:58.991241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:58.992190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:58.993042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:58.993042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:58.993042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:58.996917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 382/746] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:58.997768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.014798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.032837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3285909c-4e05-4239-8a32-c6d91d4e7e12 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.043738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.081672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.082723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.083200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.083667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.084657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:59.085158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.087041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.109269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 365/747] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:59.110066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ee08c-45d6-437f-b821-e68f22478360 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.112052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.146443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.147077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.147349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.147615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.158222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.159675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:59.176065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:59.177034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:59.178466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:59.178466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:59.178466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:59.181943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 383/748] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:59.182914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.234228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932eed2a-e668-4c6c-9860-f18621daf221 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.249257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.281216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.282978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.283502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.283952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.285067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:59.285983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.287943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.291514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d857280f-924b-49b1-aa11-53351e7c46a4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.306778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 366/749] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:59.306778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.336389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.337014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.337354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.337540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.341065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.349852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.351498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:07:59.367375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:59.368350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:07:59.369826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:07:59.369826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:07:59.369826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:59.369826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 384/750] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:59.369826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.434887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ea499-30fd-4bae-bb17-e9e3a57425d6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.475753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.476357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.476633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.476903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.477539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:59.477834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.479709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.497930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 367/751] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:59.498909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.505834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5368b9f4-515b-4ccb-b081-daaaaf81c1cc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.528629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.554682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.556111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.556811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.557232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.568974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.570935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:59.587665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:59.589012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:59.590106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:59.597805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 385/752] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:59.597805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.599865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.618522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7753bc93-0efd-4d27-a83e-b81a2598063d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.665292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.666331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.666826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.667314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.668913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:59.669730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.671644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.692463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 368/753] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:59.693750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.728628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1475098d-cbe2-454e-9466-769095eca7db None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.768312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.769139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.769479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.769815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.777900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.779700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:07:59.791978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:59.792957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:07:59.793878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:59.798519 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 386/754] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:59.799400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.817994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3cef6d-3206-4ed5-a9b5-194a921deb7a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.837144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.853108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.853717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.853993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.854292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.854976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:07:59.855277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.856690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.874091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 369/755] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:07:59.874965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.880389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:07:59.906948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a45e73-da4a-4db6-9073-5dc7070e1c53 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:07:59.942112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:07:59.942805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.943028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.943292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.950021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:07:59.951499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c06be355-d8b2-4223-b3d2-52f523e75270 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:07:59.963963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:07:59.964878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:07:59.965754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 387/756] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:07:59.966852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146490d-463b-4711-bf8f-257dd07c34ea None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:07:59.966852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146490d-463b-4711-bf8f-257dd07c34ea None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:07:59.966852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146490d-463b-4711-bf8f-257dd07c34ea None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:07:59.968088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 388/757] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:07:59 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:07:59.973274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146490d-463b-4711-bf8f-257dd07c34ea None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:07:59.991324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948d7be-9abc-44ab-8e55-0d1f31a616dd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.025468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.026074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.026383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.026653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.027368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:00.027665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:00.029090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:00.051886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 370/758] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:00.051886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:00.083695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:00.091714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146490d-463b-4711-bf8f-257dd07c34ea None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.116938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:00.129328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.129929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.130225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.130537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.131264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=bf8cf97352f244d4b87c5927ee451dd9) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:00.131574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:00.133007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ceb884-28e7-4181-ab4e-2df36535657c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:00.138689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 389/759] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:07:59 2022] GET /identity/v3/role_assignments?user.id=bf8cf97352f244d4b87c5927ee451dd9&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:00.140653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.141162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.141797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.161140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8521ffe-2aee-49dc-85ea-1d9a6eb77c67 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.166963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:00.168543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:00.199018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.199580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.199865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.200141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.200852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:00.201162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:00.202737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:00.221480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 371/760] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:00.221480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:00.268313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 390/761] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:00 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:00.269558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:00.374565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d5d46-de6c-44ee-8ecf-8f8436f1469f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.412606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:00.414150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:00.426085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb74d99d-7acc-445d-9bf4-220a6e6661da None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.455234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.455846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.456134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.456416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.465543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.466229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.466501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.466809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/a8514e96615c4785aefde8261120afe3/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.469255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=a8514e96615c4785aefde8261120afe3, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:00.470900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:00.509196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 391/762] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:00 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/a8514e96615c4785aefde8261120afe3/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:08:00.552985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:00.566460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:00.568072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:00.692369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 372/763] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:00 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:00.698564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:00.737316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:00.739918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e8933c-5460-4728-8af2-b46751ffdd03 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.790322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.790949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.791226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.791497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] PATH_INFO: `/v3/services/network` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:00.792085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:00.793464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:00.812687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:00.815337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:00.817211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 392/764] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:00 2022] GET /identity/v3/services/network => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:00.820174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:00.854757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:00.884573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57817eef-2f19-4752-8267-c7277ee4b337 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.966840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55b22d-c26b-4d61-9e80-a71478d96774 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:00.995596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:00.996381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:00.996518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:00.996700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.008486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.009355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.012180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.012634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.013430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] RBAC: Adding query filter params (name=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:01.013877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.015461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.023130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.024905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.027234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 393/765] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:00 2022] GET /identity/v3/services?name=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:01.033554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:01.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:01.061698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:01.062815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:01.062815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:01.062815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:01.065721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 373/766] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:01.069069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.164991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:01.190122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53487edb-cb37-4507-89d0-00fc0c503145 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.202228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:01.228802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0633154-a205-4ae1-ac55-6323e52d6ddb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.229485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.230451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.231020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.231291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.231930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] RBAC: Adding query filter params (type=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:01.232230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.233600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329ebf0-3a7d-41f0-9638-1940def52b75 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.239558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 394/767] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/services?type=network => generated 350 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:01.240541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444435-e001-4d81-995c-3c9a057a5a88 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.240823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444435-e001-4d81-995c-3c9a057a5a88 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.241091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59444435-e001-4d81-995c-3c9a057a5a88 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.247441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 395/768] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:01.248243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444435-e001-4d81-995c-3c9a057a5a88 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.266390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.268117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.268548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.269068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.269857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:01.270333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.271889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94bfedd-5e27-4f7e-8bae-9f349f68904d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.291068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 374/769] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:01.292754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.293195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.293617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.318709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:01.320578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:01.361099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59444435-e001-4d81-995c-3c9a057a5a88 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.401596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.402567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.402847 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.403124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.403827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] RBAC: Adding query filter params (interface=public, service_id=cd70b0ebf03b405a9e964411a843440f, region_id=RegionOne) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:01.404135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.405498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.414704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 375/770] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:01 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:01.416317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.420323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 396/771] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/endpoints?service_id=cd70b0ebf03b405a9e964411a843440f&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:01.430427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.444710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:01.534040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:01.603429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a496db3-eb2b-4c32-8074-3a37b35fc5eb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.608387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b160fbb8-91cc-46db-8144-9966feaa49bb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.650535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.651374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.651886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.652403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.653162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.654732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.661568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 397/772] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:01.695258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.695934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.696213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.696486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.711407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.712872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.727171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:01.736752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:01.737746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:01.739083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 376/773] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:01.746044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.837930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:01.913476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7c1fdc-933f-43f9-b37a-75449f326f47 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.935883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bcff2c-05b2-45e5-b6db-3f57ca19f924 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:01.953584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.954962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.955242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.955510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.956103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.957474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:01.966876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:01.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:01.968687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 398/774] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:01.975624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:01.977352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:01.995637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:01.996257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:01.996556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:01.996806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:01.997501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:01.997806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:01.999255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:02.021597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 377/775] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:02.038474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:02.100193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330cd266-0904-4c8b-b9b0-22c39708662f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:02.143542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:02.144314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.144589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.144860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.145503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:02.145803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:02.147212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:02.155941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:02.162874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 399/776] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Oct 4 10:08:01 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:02.168754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:02.276489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0838e8fc-2903-428c-8af5-ea8e35115577 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:02.339270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:02.340168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.340457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.340731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.341455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=9f385f2a341a4162af72414f4dc0ee91) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:02.341759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:02.343223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:02.350086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 378/777] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:02 2022] GET /identity/v3/role_assignments?user.id=9f385f2a341a4162af72414f4dc0ee91&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:02.394289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d50553-5ac3-4022-812c-72cf33554c22 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:02.452469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:02.453086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.453363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.453630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.454464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:02.454813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:02.456162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:02.461941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 400/778] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Oct 4 10:08:02 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:02.467779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:02.544331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:02.620954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebdf230-5c4c-44a9-bace-8ac1b220c2c0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:02.658611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:02.659145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.659428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.659698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.660403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] RBAC: Adding query filter params (interface=public, service_id=ee6ae0108f424f6583bf6d0640ba857b, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:02.660710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:02.662067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85b0d29-537c-45dd-bf22-80f29619e552 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:02.670465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 379/779] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:02 2022] GET /identity/v3/endpoints?service_id=ee6ae0108f424f6583bf6d0640ba857b&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:02.676444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:02.731736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75f27e-70be-4bff-a7be-2ba4397b7968 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.732077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75f27e-70be-4bff-a7be-2ba4397b7968 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.732330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75f27e-70be-4bff-a7be-2ba4397b7968 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.733899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 380/780] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:02.740134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.740419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.740696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.760313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:02.763802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:02.764995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:02.845153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62425d5c-f7e3-4628-8160-922d63cff251 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:02.901688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 381/781] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:02 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:02.909014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473cc39-d896-4eb4-a6a3-85beb3d4fffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:02.909559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473cc39-d896-4eb4-a6a3-85beb3d4fffe None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:02.909837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473cc39-d896-4eb4-a6a3-85beb3d4fffe None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:02.910116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473cc39-d896-4eb4-a6a3-85beb3d4fffe None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:02.910755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473cc39-d896-4eb4-a6a3-85beb3d4fffe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:02.912122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473cc39-d896-4eb4-a6a3-85beb3d4fffe None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:02.917662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 401/782] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:02 2022] GET /identity/v3/services => generated 772 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:03.179518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:03.226827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:03.227135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148f1e-267f-491f-9589-6af28edec276 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.227559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148f1e-267f-491f-9589-6af28edec276 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.227658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d148f1e-267f-491f-9589-6af28edec276 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.229204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 402/783] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:03.234542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.234823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.235194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.256880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:03.258092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:03.290809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b74705-dc03-4331-95ff-257f1800f909 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:03.326109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:03.326784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.327060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.327332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.339817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:03.343106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 403/784] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:03.347430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:03.348577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:03.349623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 382/785] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:03.356021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:03.414027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:03.459598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8749bc-6d78-4791-a2ba-35fd22e1f49a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:03.504090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:03.504945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.505368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.505785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.506621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:03.507102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:03.508611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:03.522443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 404/786] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:03.528193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:03.669897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ec85b7-5cf3-44c0-b5ee-e55caeb4eec8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:03.671456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:03.719593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:03.720152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.720425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.720693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.733694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:03.735200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:03.757085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:03.758069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:03.758974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:03.758974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:03.758974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:03.758974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 383/787] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:03.764328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:03.771434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:03.898689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9b3ce-d828-4a96-bade-54409a83c173 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:03.902096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e760c0c7-ccd1-426a-8665-b38ba97edaf1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:03.942864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:03.943506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.943833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.944113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.947594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:03.957823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:03.964401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:03.965313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:03.966157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 384/788] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:03.973213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:03.990759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:03.991376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:03.991655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:03.991991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:03.992647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:03.995094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:03.998586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.008294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:04.028247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 405/789] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:04.033438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.158571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b15472-b5a5-4598-8d3d-f758a3f45603 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.198945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.199878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.200163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.200437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.207850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8183e71-c8bb-449a-87fd-4c4b9678e9a2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.211678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.213132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:04.227361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:04.229981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:04.232224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 406/790] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:04.237743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.273538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.274155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.274454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.274724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.275386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:04.275682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.277044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.298273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 385/791] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:04.303849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.320932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:04.393495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:04.405887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb21ff67-84ee-4f72-80cb-62c00b5f5d12 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.419461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f69913f-b86e-4488-8444-4616036c4da3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.443565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.444287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.444560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.444828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.445557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:04.445861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.447552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47dec1bd-ad37-4a96-bafd-6af72076abc0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.455847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.456792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.457322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.457741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.466631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 407/792] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:04.471791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3cf8-1c51-40d1-88e2-323c93622326 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.471791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3cf8-1c51-40d1-88e2-323c93622326 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.471791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d3cf8-1c51-40d1-88e2-323c93622326 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.471791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 408/793] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:04.472922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3cf8-1c51-40d1-88e2-323c93622326 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.475087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.476966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-341c0164-df5f-4663-a0cd-4a534dd6c7a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:04.492769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:04.493704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:04.494604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:04.494604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:04.494604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:04.496679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 386/794] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:04.503135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.503690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.504161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.525369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:04.527047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:04.582870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:04.591821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d3cf8-1c51-40d1-88e2-323c93622326 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.602602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 387/795] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:04 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:04.604008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.655006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.656034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.657624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.658390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.679239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.681767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:04.711598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:04.712595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:04.713750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:04.713750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:04.713750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:04.716534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 409/796] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:04.721814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:04.730574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.810626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d28ca-a4ba-4f8a-8e46-b326f35be312 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.877548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.878595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.879180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.879649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.880518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:04.881030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.882687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:04.898642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 388/797] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:04.904792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:04.928035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b2f508-cc2a-4e1b-9165-2c95db55bce5 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:04.982295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:04.983446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:04.984208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:04.984689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:04.986070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:04.986394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:04.988219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.000711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 410/798] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:05.001335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.012782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.047901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5dbed-d0c3-4368-8732-862514a0ebe2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.065233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.083599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.084227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.084798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.084798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.092585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.094177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d7ba2022-3a4c-4ee4-bd7e-c7e2ceebe004 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:05.105062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:05.105986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:05.106950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 389/799] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:08:04 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:05.108203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099a2f-27da-42ce-9369-30cad39858c1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.108203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099a2f-27da-42ce-9369-30cad39858c1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.108203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b099a2f-27da-42ce-9369-30cad39858c1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.113939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 390/800] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:05.113939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b099a2f-27da-42ce-9369-30cad39858c1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.118756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eee153f-3386-496c-9351-3bd2ed8a554a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.154525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf09994-230e-44e6-a350-41a31405250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.155149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf09994-230e-44e6-a350-41a31405250c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.155424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf09994-230e-44e6-a350-41a31405250c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.155688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf09994-230e-44e6-a350-41a31405250c None admin] PATH_INFO: `/v3/services/network` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.156261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf09994-230e-44e6-a350-41a31405250c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.157669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf09994-230e-44e6-a350-41a31405250c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-abf09994-230e-44e6-a350-41a31405250c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:05.166398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:05.167394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:05.168789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 411/801] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:05.169587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf09994-230e-44e6-a350-41a31405250c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.229706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b099a2f-27da-42ce-9369-30cad39858c1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.239963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.267412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.268205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.268786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.269079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.271298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf09994-230e-44e6-a350-41a31405250c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.276029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.277458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e364a90c-f1e1-4241-a221-9496773e367c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:05.289874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:05.290812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:05.291743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 391/802] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:05.292252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.292969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.292969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.310561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.311045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:05.311475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.311475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.311740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.312221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:05.312462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:05.312762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.314746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.332022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 412/803] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:05.335090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.348103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.388400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 392/804] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:05.389345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.464953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1379b93-ff83-4e8a-891e-f2511c7ef66f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.494984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.503005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.503689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.504059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.504397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.505125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] RBAC: Adding query filter params (name=network) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:05.505495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.507017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.511990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 413/805] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/services?name=network => generated 124 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:05.518143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.523143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03026f32-153d-4914-a444-5e92c9857c82 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.574643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.576393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.577256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.577782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.579604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:05.580705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.583156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.604176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 393/806] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:05.607199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.658065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a785fe73-c8ef-4bea-841e-8111b7b6c43b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.667778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.696571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.697184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.697462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.697755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.746143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f05a6f-800c-4c3e-ba76-928891822ff5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.759558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.767741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.769194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.780125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.780720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.780993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.781259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.781919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] RBAC: Adding query filter params (type=network) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:05.782240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.783639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.789264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 394/807] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/services?type=network => generated 350 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:05.790001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.868197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 414/808] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:05.869027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.883110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.892190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c375ed-e445-49ed-840d-aefbe6f3ad6b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:05.925874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39af493-cefb-4f22-a0f5-8596a73cd7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:05.926567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af493-cefb-4f22-a0f5-8596a73cd7d9 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.926845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af493-cefb-4f22-a0f5-8596a73cd7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.927123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39af493-cefb-4f22-a0f5-8596a73cd7d9 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/bf8cf97352f244d4b87c5927ee451dd9/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.929029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af493-cefb-4f22-a0f5-8596a73cd7d9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=bf8cf97352f244d4b87c5927ee451dd9, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:05.930476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39af493-cefb-4f22-a0f5-8596a73cd7d9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:05.943899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 395/809] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:05 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/bf8cf97352f244d4b87c5927ee451dd9/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:08:05.945109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ff80f-f627-49ad-b06c-e6167ecde146 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:05.945396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ff80f-f627-49ad-b06c-e6167ecde146 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:05.945671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ff80f-f627-49ad-b06c-e6167ecde146 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:05.947398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 396/810] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:05.948062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ff80f-f627-49ad-b06c-e6167ecde146 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:05.988557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:05.997825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d1a2d1-fd2d-46e2-b9a0-ceedfbf1f86e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.038502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.039130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.039405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.039672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.052667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:06.058360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:06.059504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:08:06.060680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:06.065896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 415/811] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:08:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:06.067053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.087373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ff80f-f627-49ad-b06c-e6167ecde146 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.135632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64bb46d-5dc3-414d-8caa-fadf775d4cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.136501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bb46d-5dc3-414d-8caa-fadf775d4cca None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.136775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bb46d-5dc3-414d-8caa-fadf775d4cca None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.137043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64bb46d-5dc3-414d-8caa-fadf775d4cca None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.137713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bb46d-5dc3-414d-8caa-fadf775d4cca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.139095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64bb46d-5dc3-414d-8caa-fadf775d4cca None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.163191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 397/812] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:05 2022] POST /identity/v3/endpoints => generated 337 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:06.167565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.167906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.168181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.189902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:06.191221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:06.193663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:06.214842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d65ec60-dc53-47e7-b949-28708f0eb1f2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.274600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.275237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.275237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.275557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.287758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 398/813] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:06 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:06.288186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.289194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.297621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:06.325999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:06.327026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:06.327939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:06.327939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:06.327939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:06.335848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 416/814] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:06.341460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.349377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:06.420788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf6461c-696f-4028-b30c-f36eb5b4683a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.472215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.473178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.473403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c77f8c-f798-4b25-afb5-606da6a60f7b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.473877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.474403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.475504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:06.476817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.479443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.489671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:06.493911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 399/815] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:06.500252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.520326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.520974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.521309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.521559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.522177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:06.522536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.524005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.540932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 417/816] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:06.545806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.633585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:06.637312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24aae3e-1d74-4d79-b7bc-191cf3624453 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.673640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.673992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.674311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.674609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.680074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c111da3-b885-400e-a7a6-8e04d559d69d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.686647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.688123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:06.703678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:06.705268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:06.706181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:06.706181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:06.706181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:06.706181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 400/817] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:06.706181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.724158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.724791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.725069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.725341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.732376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.733844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:06.746075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:06.747020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:06.747922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:06.752033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 418/818] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:06.753461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.763975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:06.818678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d0a8ad-6a27-4ad2-a7aa-ea60efc2ef97 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.854394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.855019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.855293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.855708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.856596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.858269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.858976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c56f8-9572-4a14-87e4-3d8f6f3fc3ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:06.869372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:06.870449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:06.871298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 401/819] 10.222.0.13 () {56 vars in 1129 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:06.872165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.889589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:06.902485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:06.903090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:06.903391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:06.903668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:06.904307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:06.904605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:06.906029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:06.918858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 419/820] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:06.919919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:06.990026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf22974a-7ad9-4246-ad20-7e44ad58edd3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.007929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:07.026139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.026790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.026893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f19f3-f8d7-417e-98fc-eceeefd20819 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.027066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.027333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.028019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:07.028334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.029750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.041723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 402/821] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:07.042544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.070532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.071141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.071467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.071746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.072382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:07.072806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.074301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.083909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 420/822] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Oct 4 10:08:06 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:07.083909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.130477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:07.160103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62fe76-a439-4944-8d4f-e828fc38ef37 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.192639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fa5494-772d-49b3-9b61-f6072474aef2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.194329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.194958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.195234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.195514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.203608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.205033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:07.216867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:07.217829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 403/823] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:07.219028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.237306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.237910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.238183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.238482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.239202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=a8514e96615c4785aefde8261120afe3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:07.239506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.240982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.247154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 421/824] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/role_assignments?user.id=a8514e96615c4785aefde8261120afe3&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:07.255254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.269099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:07.367530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85462b5-62c1-4d0a-b67e-8e81ca626618 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.390836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0badfa02-d3b8-46d4-8eb3-c8808e0a6885 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.403385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:07.422557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.424265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.424554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.424834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.425471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:07.425775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.427172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.433842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 404/825] 10.222.0.13 () {56 vars in 1124 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:07.438259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.438950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.439161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.439427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.440166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:07.440472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.441917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.444442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.463893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 422/826] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:07.469936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.562755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:07.568236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb1a28-bc83-4530-ac18-82d9f33027cb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.605681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bd8bd6-6396-4932-b7a2-4c9f0c7381cf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.611774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.612759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.613241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.613712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.614650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.616710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.650486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 405/827] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:07.659065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.659797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.660073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.660345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.672481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.673978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:07.690884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:07.691893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:07.692782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:07.692782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:07.692782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:07.692782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 423/828] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:07.697986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.815041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd014470-0519-4192-9e01-621d71eeef8e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:07.853652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:07.854285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:07.854564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:07.854836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:07.855470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:07.855768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:07.857117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:07.870191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 406/829] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:07.876242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:07.896180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:07.981312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4c7f70-1394-4c76-bde2-94023c4d3fce None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:08.023150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:08.023784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.024121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.024400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.031727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:08.034375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a02673ce-f2cd-4800-81f1-5f4868619661 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:08.046233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:08.047213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:08.048108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 424/830] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:08.054952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:08.076419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:08.165486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f193f001-cb8a-4ef5-a7dd-54220074f59c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:08.179800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a9b81-ab5b-4062-88c1-1efef4a801fa None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.180100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a9b81-ab5b-4062-88c1-1efef4a801fa None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.180351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4a9b81-ab5b-4062-88c1-1efef4a801fa None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.182251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 425/831] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:08.191528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.191819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.192082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.207155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:08.207785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.208064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.208337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.209025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:08.209330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:08.210887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:08.211675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:08.212983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:08.226416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 407/832] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:08.232093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:08.306849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 426/833] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:08 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:08.316442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:08.340620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5696f004-06b5-416b-80ee-cc1e275f1367 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:08.374978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:08.376264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.376549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.376820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/9f385f2a341a4162af72414f4dc0ee91/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.391536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=9f385f2a341a4162af72414f4dc0ee91, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:08.393100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:08.411849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 408/834] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:08 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/9f385f2a341a4162af72414f4dc0ee91/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:08:08.579927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:08.589137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:08.694456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bae99be-6f0f-4581-8a3b-48ca2704ef10 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:08.729695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:08.730351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.730627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.730894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.741797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:08.746603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:08.747516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:08.748327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:08.752615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 427/835] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:08.753993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:08.854526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39548a86-657f-4161-9b47-baa6d1fe1022 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:08.887304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:08.893646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:08.894289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:08.894608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:08.894889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:08.895555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:08.895859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:08.897230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:08.906161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 409/836] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:08.912472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:08.986635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:09.027025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec102713-33d2-4d5d-aa10-ac91613e1920 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:09.066778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:09.067403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.067681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.067951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.078368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:09.080476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:09.096874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:09.099042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:09.100252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:09.100252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:09.100252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:09.100252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 428/837] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:09.103820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:09.211101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b5d47-2604-47b0-9bc8-bb00db9b8b8c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:09.250099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:09.250847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.251191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.251469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.252114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:09.252416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:09.253772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:09.266496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 410/838] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:09.272063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:09.401635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974a9091-ca26-4e4c-a600-a974477554d4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:09.424679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:09.442395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:09.443018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.443288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.443557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.451168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:09.452603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:09.468163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:09.469117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:09.470758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 429/839] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:09.476448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:09.529654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:09.586623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d856da30-b567-450e-978b-ef6bd634bd5e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:09.624994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:09.625756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.626413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.626709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.627467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:09.627769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:09.629261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ea38b-f92c-474a-a209-c1f9fad490c2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:09.649952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 411/840] 10.222.0.13 () {56 vars in 1135 bytes} [Tue Oct 4 10:08:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:09.655490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:09.747344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31fd97-2b15-4be4-bccc-09b89828b8dc None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.747639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31fd97-2b15-4be4-bccc-09b89828b8dc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.747805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f31fd97-2b15-4be4-bccc-09b89828b8dc None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.749510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 412/841] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:09.754806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.755355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.755533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.774479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:09.775792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:09.780645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ad751-f666-4aae-a941-dd98e876b224 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:09.816801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68abcfca-db85-48c5-91ba-ebec6fb62592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:09.817408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abcfca-db85-48c5-91ba-ebec6fb62592 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:09.817683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abcfca-db85-48c5-91ba-ebec6fb62592 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:09.817950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68abcfca-db85-48c5-91ba-ebec6fb62592 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:09.858476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 413/842] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:09 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:09.867971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:09.889798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abcfca-db85-48c5-91ba-ebec6fb62592 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:09.891295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68abcfca-db85-48c5-91ba-ebec6fb62592 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:09.996441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 430/843] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:09 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:10.003422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.043690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.047586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929cfc-8d39-4e6d-8aaa-a121fb23dd75 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.047923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929cfc-8d39-4e6d-8aaa-a121fb23dd75 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.048195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7929cfc-8d39-4e6d-8aaa-a121fb23dd75 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.049863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 431/844] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:10.054940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.055217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.055492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.073936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:10.075148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:10.118394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4df6df-112f-4bb1-aa5d-6a02988e54ec None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.149488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 432/845] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:10.152404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.155451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.156397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.156874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.157295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.164860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.169361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.171726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:10.190284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:10.191260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:10.192159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:10.192159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:10.192159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:10.194395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 414/846] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:10.199756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.279674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dd036-63aa-471d-a97b-d1d1b0745061 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.299128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.312694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02edf6ff-172e-47b8-bd38-0ef4b621eae5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.325935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.326616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.326893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.327165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.327757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.330013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:10.340115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:10.341055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:10.341948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:10.346707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 433/847] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:10.348059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.354390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.355273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.355756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.356222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.357056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:10.357576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.359299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.373985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 415/848] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:10.381261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.440227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.474777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3619d27f-35dd-4de5-a977-fa8fdc7e50a6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.485021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cca82a-b011-4b25-821f-7389682b23c2 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.512230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.512831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.513107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.513375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.514002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:10.514325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.515674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.520947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 434/849] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:10.521713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.529883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.531272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.531787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.533164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.543391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.545888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:10.561754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:10.562776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:10.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 416/850] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:10.567170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.582987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.637730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe69088-ee06-466d-a531-7a640677ce87 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.675839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.676472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.676758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.677030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.677616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.679010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.681599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.686013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f8e45-f45d-4ce1-825b-1886b7e42b3e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:10.689458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:10.690653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:10.692049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 435/851] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:10.692832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.721777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.722411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.722684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.722954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.723574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:10.723873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.725226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7749972-3d7b-4481-acd8-f531a71bc546 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.729501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 417/852] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:10.730673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c306990-c06e-408e-8f73-d2feb6c0e127 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.730951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c306990-c06e-408e-8f73-d2feb6c0e127 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.731297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c306990-c06e-408e-8f73-d2feb6c0e127 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.732629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 418/853] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:10 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:10.733248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c306990-c06e-408e-8f73-d2feb6c0e127 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.810874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c37f725-bd93-4fd8-ade0-22fb643f1e59 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.817091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.851784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.852403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.852678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.852953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.853648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:10.853958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.854433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c306990-c06e-408e-8f73-d2feb6c0e127 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:10.855467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c18707d-dfed-4fa2-902c-e9d3ba2889bc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.869089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 436/854] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:10.870797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.871278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.871730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.890610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:10.892652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:10.898309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:10.899590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:10.899590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:10.899590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:10.900070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:10.900367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:10.901734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:10.906976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 419/855] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:10.907692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:10.961596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:10.974792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 437/856] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:10 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:10.975698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:11.023396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23bcfb-7e80-44e6-bf04-9995db94baa7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:11.069467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:11.070105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:11.070403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:11.070795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:11.071636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=bf8cf97352f244d4b87c5927ee451dd9) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:11.071947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:11.073377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:11.081055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 420/857] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/role_assignments?user.id=bf8cf97352f244d4b87c5927ee451dd9&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:11.081815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:11.112662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3069e82c-caf0-4628-a2d1-99c3bc6f17e7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:11.156622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:11.158236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:11.158694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:11.158969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:11.159697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:11.160000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:11.161430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:11.167203 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 438/858] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:10 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:11.269306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:11.407645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc92788-2782-45ae-9b3a-ddfbc5da240e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:11.443794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:11.444438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:11.444715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:11.444988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:11.456932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:11.462483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:11.464746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:11.466477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 421/859] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:08:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:11.470391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:11.532052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:11.582539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be870c72-dd5c-4679-a8ae-9fec9040d01a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:11.628371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:11.628997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:11.629387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:11.629610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:11.630446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:11.630822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:11.632302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:11.646334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 439/860] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:08:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:11.648478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:11.742887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:11.804933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72de5d9f-1707-495e-86a6-dd2e08ba6290 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:11.854129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:11.856000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:11.856303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:11.856605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:11.870783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:11.873037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:11.893879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:11.895998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:11.897750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:11.897750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:11.897750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:11.897750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 422/861] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:11.902825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:12.051238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7575c0e0-cb19-4d5d-b554-39aa46687d89 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:12.069022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:12.089697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:12.090334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:12.090613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:12.090942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:12.091594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:12.091951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:12.093331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:12.107274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 440/862] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:12.113439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:12.220938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808f3172-53b0-4ff9-903f-208553e7019a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:12.234270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:12.258231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:12.258870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:12.259151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:12.259431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:12.267971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:12.269399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:12.283978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:12.284903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:12.285838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:12.288361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 423/863] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:08:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:12.294436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:12.401135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb69692-45d1-4619-9a60-b917ecb69d58 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:12.436811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:12.437843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:12.438481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:12.438955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:12.439922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:12.440427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:12.442099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:12.461065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 441/864] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Oct 4 10:08:12 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:12.466993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:12.574399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d68b9-571d-467d-a4b9-02d10e371552 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:12.609055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:12.610185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:12.610480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:12.610750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:12.653823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:12.687125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:12.689058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:12.792503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 424/865] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:12 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:12.798277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:12.833490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:12.902832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cb5ab5-a6ab-4297-8dcd-5faa704230f7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:12.943343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:12.944245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:12.944669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:12.945099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:12.956782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:12.958491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-0968c616-0873-47e6-a976-b0c1ed273343 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:12.974380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:12.975564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:12.976668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:12.976668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:12.976668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:12.978378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 442/866] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:12.987315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:13.036304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569c84d-9a43-4a21-95fe-10d217453155 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.036576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569c84d-9a43-4a21-95fe-10d217453155 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.036844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569c84d-9a43-4a21-95fe-10d217453155 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.038191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 443/867] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:13.043059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.043333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.043603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.063545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:13.063898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:13.065493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:13.102450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448a0792-5b95-4486-b847-2b15cbff2a48 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:13.143835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:13.144446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.144718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.144987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.145622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:13.145917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:13.147302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:13.149473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 444/868] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:13 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:13.171889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 425/869] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:13.174320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:13.193709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:13.286099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05029a8f-cd81-4604-8853-f64310cecec7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:13.321534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:13.322172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.322481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.322811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.330859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:13.332365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:13.345309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:13.346191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:13.347525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 445/870] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:13.354271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:13.445686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:13.468388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b59aec-8067-41d2-b6ab-b5a530579a26 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:13.502081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:13.508998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:13.510041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.510552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.511042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.511925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:13.512440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:13.514108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06069dd8-35d6-45b3-8565-c9a006ced321 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:13.532084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:13.532290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 426/871] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:13.534237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3739e6-3bde-4188-980d-9ddb7ed437b8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.534715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3739e6-3bde-4188-980d-9ddb7ed437b8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.534982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3739e6-3bde-4188-980d-9ddb7ed437b8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.540412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 427/872] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:13.540412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3739e6-3bde-4188-980d-9ddb7ed437b8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:13.564725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ea312e-6d23-429c-a8b5-643ad142be28 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:13.609413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aaf7ad-cbe6-49be-ba37-9d78d02df383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:13.611152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaf7ad-cbe6-49be-ba37-9d78d02df383 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.611493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaf7ad-cbe6-49be-ba37-9d78d02df383 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.611768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aaf7ad-cbe6-49be-ba37-9d78d02df383 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.612437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aaf7ad-cbe6-49be-ba37-9d78d02df383 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:13.613809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aaf7ad-cbe6-49be-ba37-9d78d02df383 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:13.634364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 446/873] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:08:13 2022] POST /identity/v3/services => generated 269 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:13.636627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.637166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.637923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.664405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:13.664605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3739e6-3bde-4188-980d-9ddb7ed437b8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:13.666392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:13.709369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:13.709898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.710261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.710487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.711282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=9f385f2a341a4162af72414f4dc0ee91) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:13.711541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:13.713018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47c129e-032f-4cdc-9915-d8efb785afdc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:13.720512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 428/874] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:13 2022] GET /identity/v3/role_assignments?user.id=9f385f2a341a4162af72414f4dc0ee91&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:13.721870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5def-1998-4c91-8532-eda40511f5e1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.722164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5def-1998-4c91-8532-eda40511f5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.722472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15e5def-1998-4c91-8532-eda40511f5e1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.726373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 429/875] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:13.729618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:13.730289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:13.730769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:13.755718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:13.758416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:13.762979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 447/876] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:13 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:13.819933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:13.848439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 430/877] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:13 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:14.054401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:14.124035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:14.154987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:14.162762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c499c3f4-4958-4797-924d-0a7b95710181 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:14.199360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:14.200363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:14.200838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:14.201317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:14.202118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:14.204563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:14.214251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:14.215493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:14.216531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:14.223721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 448/878] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:14.225156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:14.269158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02bc61-19bb-49f2-b3e8-42df1b1dd265 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:14.307682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:14.308461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:14.308813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:14.309245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:14.310022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:14.311586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:14.334365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e45639-4720-43d9-9b6a-b1d4c14fcabb None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:14.336573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 431/879] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:08:14 2022] POST /identity/v3/services => generated 251 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:14.368765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:14.381728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:14.382367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:14.382646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:14.382916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:14.383549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:14.383846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:14.385215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dbbaa5-cf53-4a8d-98a5-e89198e78c35 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:14.395672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 449/880] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:14.397126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:14.447711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:14.514783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3a3b6-58c2-4fba-8ff4-5ee82a7664fe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:14.552313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:14.552930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:14.553209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:14.553485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:14.554126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:14.554452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:14.555808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c890a2-33b3-4d4e-a46f-485d6c855f02 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:14.562448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 432/881] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:14 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:15.948150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca516a0-faa5-4cc1-9b28-df0c39916008 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:15.949081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca516a0-faa5-4cc1-9b28-df0c39916008 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:15.949590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca516a0-faa5-4cc1-9b28-df0c39916008 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:15.951410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 450/882] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:15.957366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa7773-4c90-4abe-9aaf-9bcea25e88a5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:15.958193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa7773-4c90-4abe-9aaf-9bcea25e88a5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:15.958703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aaa7773-4c90-4abe-9aaf-9bcea25e88a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:15.979433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2aaa7773-4c90-4abe-9aaf-9bcea25e88a5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:15.981035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2aaa7773-4c90-4abe-9aaf-9bcea25e88a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:16.055131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 433/883] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:15 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:16.102781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913f1b1-1e0b-4353-9601-7f3fe950b53a None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.103089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913f1b1-1e0b-4353-9601-7f3fe950b53a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.103321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6913f1b1-1e0b-4353-9601-7f3fe950b53a None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.104921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 451/884] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:16.110944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.111221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.111491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.129693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:16.130893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:16.160349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:16.201954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 434/885] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:16.330411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913f1b1-1e0b-4353-9601-7f3fe950b53a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.436834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6913f1b1-1e0b-4353-9601-7f3fe950b53a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:16.475419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.476108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:16.476764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.477043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.477346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.478043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:16.479616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:16.489855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:16.490902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:16.491774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 452/886] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:16.497137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.576114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:16.596679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b819d93f-b8d8-4edb-8aeb-5a2a56dc9d99 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:16.630021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6492d5c-ff3c-442d-8a8f-a31e01c69098 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:16.640655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb36160-5d0e-4572-9ca4-baeb78f96406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:16.641313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36160-5d0e-4572-9ca4-baeb78f96406 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.641595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36160-5d0e-4572-9ca4-baeb78f96406 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.641870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb36160-5d0e-4572-9ca4-baeb78f96406 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.652588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb36160-5d0e-4572-9ca4-baeb78f96406 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-adb36160-5d0e-4572-9ca4-baeb78f96406 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:16.657959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:16.659051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:16.660267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:16.663907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 435/887] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:16.665085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344488c-436e-4f10-b999-2c8d980cfc87 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.665362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344488c-436e-4f10-b999-2c8d980cfc87 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.665629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344488c-436e-4f10-b999-2c8d980cfc87 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.667605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 436/888] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:16.668149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344488c-436e-4f10-b999-2c8d980cfc87 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.676656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:16.677269 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.677916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.677916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.678610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:16.678911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:16.680304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57d73ec-143d-4a3e-a8f1-e4b2479c2dde None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:16.685668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 453/889] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:16.686805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.687087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.687359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.704663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:16.706762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:16.708253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:16.779597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 454/890] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:16.780449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.799910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344488c-436e-4f10-b999-2c8d980cfc87 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:16.835563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:16.841906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:16.843253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.844586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.845095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.845972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:16.846530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:16.848059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4928a250-bb7c-488f-bafe-da8cc17c898b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:16.858254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 437/891] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:16.859512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c42dc-e7bf-4745-bf33-0ade26c9cd95 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.859794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c42dc-e7bf-4745-bf33-0ade26c9cd95 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.860063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c42dc-e7bf-4745-bf33-0ade26c9cd95 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.865477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 438/892] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:16.866502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.866779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.867048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.892531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:16.893724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:16.900826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1dd34-c421-4173-83fe-fdc3a6a5312c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:16.936510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:16.937354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:16.937983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:16.938336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:16.938971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:16.939271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:16.940623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:16.950215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 455/893] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:16.951047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.970276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 439/894] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:16 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:16.970276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:16.991537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.078026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ebff8c8-1a63-4728-976e-1663793e0238 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.081654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443740fc-eeec-4801-bb5c-41bdc4667653 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.096396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.118132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.119143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.119529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.119830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.120148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.120426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.120701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.120815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.121846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] RBAC: Adding query filter params (interface=public, service_id=fd89f70e6b4045e7a74731e8f054c13c, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:17.122367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.123901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.129119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 440/895] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/endpoints?service_id=fd89f70e6b4045e7a74731e8f054c13c&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:17.130104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.136063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.137505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:17.153228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:17.154304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:17.155289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:17.155289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:17.155289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:17.155289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 456/896] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:17.155289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.222630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.241874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b37e0c-d8e7-4e3f-8195-8b9535586877 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.258288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b21363-41b4-466b-9711-0a3eb193c0ab None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.277693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.278353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.278631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.278902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.279478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.280893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.291281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.291878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.292149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.292416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:17.292560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:17.293591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:17.294932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:17.295698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.295785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 441/897] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:17.295785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.295915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.302434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 457/898] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/services => generated 1268 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:08:17.303453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.337145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.415126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd72688-393a-495a-a7c5-0b0b74615f2d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.419214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe1f85-ce7f-4227-807a-e6b7f6fc0d6f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.451718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.452341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.452619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.452894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.453582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:17.453883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.455323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.459562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.460495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.460975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.461441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.462492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.464150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.473545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 442/899] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:17.474515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.481864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 458/900] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:08:17 2022] POST /identity/v3/services => generated 241 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:17.482599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.486321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.623330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37a2c9-8467-4b60-82c9-41c3c751a0f6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.626233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.653546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af8a07a-fe46-4434-a87b-f56252210424 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.678676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.679299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.679573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.679841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.681071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:17.681377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.683188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.689488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 443/901] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:17.696036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.702292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.702925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.703728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.704454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.713420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.714905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:17.728101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:17.729102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:17.730042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 459/902] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:17.735398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.784840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:17.823279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aac263-8baf-48eb-b836-449b3fea75da None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.865766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.867120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.868039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.868822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.869859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:17.870397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.872538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.873617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545ba637-e5f1-43d3-bed3-b03db0eb2153 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:17.884441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 444/903] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:17.886100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.921521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:17.922859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:17.923477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:17.923849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:17.925150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:17.925866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:17.928087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:17.952590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 460/904] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:17.959799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:17.965322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:18.024744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d1e27-bef4-4c23-929a-500fa5575a75 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:18.067038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:18.067768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:18.068282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:18.068656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:18.069372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] RBAC: Adding query filter params (interface=public, service_id=6e1bd087fea0454b99213c9dff5ae134, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:18.069680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:18.071790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:18.073972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:18.085297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 445/905] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/endpoints?service_id=6e1bd087fea0454b99213c9dff5ae134&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:18.085480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762677a-4b11-49cc-882a-a64fa27bc01c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:18.087619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:18.122586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:18.123555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:18.124089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:18.124635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:18.199762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:18.201733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:18.204087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:18.222618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2188031-496a-4948-9c17-effc0b5ca518 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:18.270049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:18.270695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:18.271379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:18.271668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:18.272281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:18.274180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:18.286241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 446/906] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:18 2022] GET /identity/v3/services => generated 1497 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:08:18.328855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 461/907] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:17 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:18.334549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:18.407882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:18.451036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e53257c-c39b-4f82-98dd-8d6dc7a4316a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:18.490935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:18.491576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:18.491852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:18.492190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:18.503651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:18.505089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:18.520657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:18.521838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:18.523165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:18.523165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:18.523165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:18.523165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 447/908] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:18.529118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:18.643168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a6cd9-4659-4a78-9c45-9b34ab1b268b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:18.644851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:18.681643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:18.682168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:18.682474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:18.682804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:18.683455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:18.683861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:18.685223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:18.697406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 462/909] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:18.703699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:18.807513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c401426-1152-4d0e-97cd-c0f6c0d9fa1c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:18.822477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:18.847712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:18.847712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:18.847712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:18.847712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:18.854798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:18.856475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:18.868907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:18.870011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:18.871120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:18.873095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 448/910] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:18.879575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:18.987802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c1c69e-1b52-4681-b3f0-490b661a05d3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:19.023621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:19.024231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:19.024508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:19.024779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:19.025532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:19.025842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:19.027305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eced6333-9756-4111-a2b9-777cd4dd19a7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:19.040741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 463/911] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:19.047730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:19.152359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526aae42-8bf0-4512-9543-21caa485ad8b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:19.188510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:19.189083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:19.189348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:19.189622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:19.190380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] RBAC: Adding query filter params (role.id=b98f044251bf4fb4ab172eb56bedeb2f, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=9f385f2a341a4162af72414f4dc0ee91) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:19.190687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:19.192113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabdfd3-c2a0-477f-bd7a-3268f492525d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:19.198180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 449/912] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:19 2022] GET /identity/v3/role_assignments?user.id=9f385f2a341a4162af72414f4dc0ee91&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=b98f044251bf4fb4ab172eb56bedeb2f => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:19.526905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ede87-7473-463c-b78e-96f35d70d9ad None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:19.527299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ede87-7473-463c-b78e-96f35d70d9ad None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:19.527474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5ede87-7473-463c-b78e-96f35d70d9ad None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:19.529056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 464/913] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:19.535231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcbd23-49ba-4a20-8814-80f649cbc9ae None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:19.535511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcbd23-49ba-4a20-8814-80f649cbc9ae None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:19.535784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbcbd23-49ba-4a20-8814-80f649cbc9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:19.554695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ffbcbd23-49ba-4a20-8814-80f649cbc9ae None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:19.555879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ffbcbd23-49ba-4a20-8814-80f649cbc9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:19.586037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:19.626397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 450/914] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:19.806705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a80f4-6043-4bd7-bb73-2a6f8115fe95 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:19.807001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a80f4-6043-4bd7-bb73-2a6f8115fe95 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:19.807204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a80f4-6043-4bd7-bb73-2a6f8115fe95 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:19.808762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 465/915] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:19.814637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:19.815161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:19.815698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:19.834087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:19.835550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:19.906026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 451/916] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:19 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:19.919330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a80f4-6043-4bd7-bb73-2a6f8115fe95 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:19.944953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:20.026859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8a80f4-6043-4bd7-bb73-2a6f8115fe95 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.065425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.065984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.066282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.066556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.067147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.068507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:20.077552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:20.078562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:20.079433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 466/917] 10.222.0.13 () {56 vars in 1132 bytes} [Tue Oct 4 10:08:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:20.085067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.186508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:20.194437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c3e02e-2c6e-4591-9672-8bf831a0ec13 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.200453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.242006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.242652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.242928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.243198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.243825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:20.244123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.245914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.252153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 452/918] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:20.258080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.295716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:20.314676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6805e39b-7033-4d04-959a-eb7fe2a59315 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.351680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.352659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.353237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.353718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.354681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.356336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:20.367059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:20.368114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:20.368948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:20.376977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8a930f-01d0-4be0-8da9-0cea91274ada None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.377329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 467/919] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:20.378445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.414179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.414816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.415094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.415365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.415994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:20.416293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.417812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.423669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 453/920] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:20.429193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.457191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:20.502910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ed1fbe-4ffe-445b-89aa-36ea7631092e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.539150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b5a7e5-678e-4d18-8f5e-7f5497030b4f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.542434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.543261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.543731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.544215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.545034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:20.545532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.547326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.557661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 468/921] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:20.558948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.575147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eb427b-c7c9-4a4e-8542-581dbb80d7bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.575814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb427b-c7c9-4a4e-8542-581dbb80d7bd None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.576090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb427b-c7c9-4a4e-8542-581dbb80d7bd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.576360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eb427b-c7c9-4a4e-8542-581dbb80d7bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.577072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb427b-c7c9-4a4e-8542-581dbb80d7bd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.578411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:20.578593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eb427b-c7c9-4a4e-8542-581dbb80d7bd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.603247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 454/922] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:20 2022] POST /identity/v3/endpoints => generated 347 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:20.604925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4d7f8-79ff-4ac2-97d4-308f10cdd983 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.604925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4d7f8-79ff-4ac2-97d4-308f10cdd983 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.605119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a4d7f8-79ff-4ac2-97d4-308f10cdd983 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.606544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 455/923] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:20.619157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.619157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.619157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.646868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:20.648501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:20.693887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28c8f03-6ada-4e94-ab91-5e7448711d3d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.740178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 456/924] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:20 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:20.743521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.744318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.744740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.745150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.745953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:20.746458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.747997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.758331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 469/925] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:20.760488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:20.773052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:20.866887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de783861-5c31-455d-839e-dd4d70f224c2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:20.901227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:20.902097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:20.902614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:20.903027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:20.903887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] RBAC: Adding query filter params (interface=public, service_id=18c858fd034d49bb8fed123d3dd98577, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:20.904344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:20.905845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:20.911711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 457/926] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/endpoints?service_id=18c858fd034d49bb8fed123d3dd98577&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:20.917456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:21.010577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:21.013527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:21.021604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69328c4c-985e-446a-bfc3-d22d04126348 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:21.065526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:21.066157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.066460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.066690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.067305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:21.069132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:21.077055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 470/927] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:20 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:08:21.094169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:21.129100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f064b2f-47a4-43d9-8907-3240b1d514c7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:21.167809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:21.168867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.169369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.169843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.170712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:21.172642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:21.183416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:21.184470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 04 10:08:21.185300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:21.188985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 458/928] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:21 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:21.195174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:21.312104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4cfd09-868f-4ebc-8377-5df02c0b4b81 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:21.351722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:21.352771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.353262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.353772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.354697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:21.355294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:21.357045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fba313-ecc3-4212-9a5c-226321537e29 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:21.363937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 471/929] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:21 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:21.369993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:21.461551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87239d2d-46f6-4b74-ba3c-491d8e8af025 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.462177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87239d2d-46f6-4b74-ba3c-491d8e8af025 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.463098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87239d2d-46f6-4b74-ba3c-491d8e8af025 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.465206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 472/930] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:21.471193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.471473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.471745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.484676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75df527d-e760-47b1-b8aa-7d7a13a59f52 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:21.485848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:21.492193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:21.493706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:21.522158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:21.522801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.523082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.523355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.523984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:21.524284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:21.525645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:21.527750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:21.531813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 459/931] 10.222.0.13 () {56 vars in 1114 bytes} [Tue Oct 4 10:08:21 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:21.537443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:21.578503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 473/932] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:21 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:21.648585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79069b34-3f20-48fe-aac2-dd048917ddf3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:21.683707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:21.684362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:21.684638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:21.684962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:21.685643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:21.687024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:21.710876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 460/933] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:21 2022] POST /identity/v3/endpoints => generated 346 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:21.861777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:21.968995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b6b7f4-368d-4933-b2d6-d50619323ba1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:22.006361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:22.012015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:22.012912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.013446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.013918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.025340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:22.030334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:22.031331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:22.032160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:22.034922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 474/934] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Oct 4 10:08:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:22.041041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:22.103654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:22.151840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcccb77-7e00-4bf0-807f-3041d69ee209 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:22.188034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:22.189036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.189539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.190021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.190945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:22.191438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:22.193021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:22.201522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 461/935] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:08:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:22.207881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:22.312178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d01c7c7-127b-4735-b345-3a22d5c5a122 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:22.347840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:22.348465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.348739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.349010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.359959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:22.361584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:22.376611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:22.377662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:22.378610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:22.378610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:22.378610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:22.378610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 475/936] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:22.383663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:22.487974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663c840-d3b1-41df-98e9-180d0bdd0eec None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:22.506147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:22.524833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:22.525448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.525727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.526042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.526776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:22.527077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:22.528438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:22.543305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 462/937] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:22.548989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:22.606517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:22.661775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5f68ee-f5ce-498c-a102-8c0649ac66ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:22.697945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:22.698606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.698880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.699149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.707411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:22.708843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:22.721272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:22.722220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:22.723168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 476/938] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:08:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:22.727995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:22.839827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7fa5ba-9efc-4554-86cd-7c52a2b440ab None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:22.875461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:22.876377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.876856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.877309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.878241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:22.878727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:22.880312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8243ca6-a8e2-48d6-9fb9-461a94f6f547 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:22.898795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 463/939] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Oct 4 10:08:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:22.904071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:22.911239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa53fe5-da57-4267-8218-6412a40e7ef9 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.911527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa53fe5-da57-4267-8218-6412a40e7ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.911805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa53fe5-da57-4267-8218-6412a40e7ef9 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.913369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 464/940] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:22.918385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:22.918665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:22.918938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:22.937170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:22.938504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:22.971949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.014459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 465/941] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:22 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:23.038835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37744039-546f-4c1d-b428-f08615319b66 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.041548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.076378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.077444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.078583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.079077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.090103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.092087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:23.108615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:23.110539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:23.111435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:23.111435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:23.111435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:23.112637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 477/942] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:23.118706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.222529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87381ad8-1524-42ab-a1b9-a8b10ca0bc95 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.262223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.263151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.263680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.264157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.265012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:23.265565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.267232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.279384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 466/943] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:23.285684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.309290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.337767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.411283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b238f139-e811-4f20-af50-4f757ccc2a94 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.433407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d78136-26e9-4301-b96a-7673e4e01146 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.454262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.454962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.455240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.455555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.465004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.466996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.473202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.474161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.474842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.475427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.476206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.477775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-7c944466-68c2-45b8-a13a-b1b4da7d2686 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:23.480177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:23.481031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:23.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 478/944] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:23.482881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0efa5b-8635-4ae9-9a8a-6882a2557ace None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.482881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0efa5b-8635-4ae9-9a8a-6882a2557ace None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.483119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0efa5b-8635-4ae9-9a8a-6882a2557ace None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.484660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 479/945] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:23.487867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0efa5b-8635-4ae9-9a8a-6882a2557ace None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f8d70114-452d-42d7-a600-b5f0d0546521 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:23.491040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:23.492041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:23.492855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:23.493603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 467/946] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/services/compute => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:23.495026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.495026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.495026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.512597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.516664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:23.517908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:23.586026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 468/947] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:23 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:23.587126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.614427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.621174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0efa5b-8635-4ae9-9a8a-6882a2557ace None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.664783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.665458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.665986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.666388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.667524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:23.668195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.669894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.685095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 480/948] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:23.691008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.715687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef20c7bb-2d47-4ab9-9d05-fd739077b538 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.753602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.754576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.755075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.755485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.756272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:23.756723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.758412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.763950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 469/949] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:23.771124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.812350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd825cb-6d86-45ab-8e56-ba0ee1fbee74 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.826576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.870229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.870229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.870229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.870229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/9f385f2a341a4162af72414f4dc0ee91/roles/b98f044251bf4fb4ab172eb56bedeb2f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.870645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=9f385f2a341a4162af72414f4dc0ee91, role_id=b98f044251bf4fb4ab172eb56bedeb2f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.870645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.880118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:23.884885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 481/950] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:23 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/9f385f2a341a4162af72414f4dc0ee91/roles/b98f044251bf4fb4ab172eb56bedeb2f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:08:23.885699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:23.902464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fe4256-c24b-4605-8a46-05d0607c98f5 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:23.949267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:23.950326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:23.951076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:23.951597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:23.953934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:23.954508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:23.956226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:23.969208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 470/951] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:24.012682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e131edb0-b679-4fbd-a444-bfce53440943 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:24.057269 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:24.057925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.058295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.058572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.059174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:24.060612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:24.065416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a5ce928f-0c9b-4ca7-a38b-d09da8245fd0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:24.075217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:24.076279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.077093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 482/952] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:23 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:24.084174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:24.111025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db0b87-0c1e-4422-8f0d-1397bc871d12 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.111598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db0b87-0c1e-4422-8f0d-1397bc871d12 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.112322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9db0b87-0c1e-4422-8f0d-1397bc871d12 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.113927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 483/953] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:24.121993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.122541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.122966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.144776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:24.146371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:24.220506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1a38d-bfe2-4190-b17d-4f266c6f875f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:24.235424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 484/954] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:24 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:24.255453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:24.256161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.256502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.256834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.257535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:24.257886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:24.259348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:24.264463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 471/955] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:24 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:24.270779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:24.379044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30861889-d9cf-4bb8-b13f-72e62328b662 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:24.409854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:24.416124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:24.416903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.417435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.417713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.418380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:24.418683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:24.420048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:24.425811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 485/956] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:24 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:24.431625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:24.448817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:24.511592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:24.552980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b570231e-590b-469f-b588-a6f343210a52 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:24.592377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:24.593409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.594601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.595267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.596590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:24.599298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:24.639606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 472/957] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:24 2022] POST /identity/v3/endpoints => generated 349 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:24.645315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d84910-15c9-4a68-9fdb-7a3a0db9377c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:24.691593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:24.692201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.692473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.692771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.693383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:24.694884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:24.697668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING keystone.server.flask.common [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 10:08:24.721230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.sql.core [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 10:08:24.721230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 10:08:24.721230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [parameters: {'id': 'e3940fa8306c47fea5d3e878c6f1838f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 10:08:24.721230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81361) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3940fa8306c47fea5d3e878c6f1838f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3940fa8306c47fea5d3e878c6f1838f. Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:08:24.740683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result.read() Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 04 10:08:24.741621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application yield resource Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise self.value Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application session.commit() Oct 04 10:08:24.743310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.session.flush() Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application compat.raise_( Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 10:08:24.744465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 10:08:24.745386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application result.read() Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:08:24.746170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [parameters: {'id': 'e3940fa8306c47fea5d3e878c6f1838f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.749187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:24.752331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id e3940fa8306c47fea5d3e878c6f1838f. Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:24.755824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 486/958] 10.222.0.13 () {60 vars in 1140 bytes} [Tue Oct 4 10:08:24 2022] POST /identity/v3/projects => generated 262 bytes in 231 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:24.757947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:24.839263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:24.877402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8c7830-36a2-4fc7-b129-f19ada2250b6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:24.916875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:24.917905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:24.918442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:24.918937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:24.927522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:24.929285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-342ea28d-e6e7-4b73-a0a6-b9d3dfaf18ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:24.943493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:24.944446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:24.945456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:24.948116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 473/959] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:24.954230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:25.004736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:25.060031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5a73e-bdda-4cee-9099-423f9256155a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:25.100157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:25.101134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:25.101631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:25.102103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:25.103039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:25.103565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:25.105758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088f7f69-27d5-476d-94d7-b38bcfd9ba7a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:25.132584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 487/960] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:24 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:26.227407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd44544-7e2d-498c-95f3-9d10e928726c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.234434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd44544-7e2d-498c-95f3-9d10e928726c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.234434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd44544-7e2d-498c-95f3-9d10e928726c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.234434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 474/961] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:26.235248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.235660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.235792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.255156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:26.256601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dede7f-b00a-4cbc-ac63-62660472f3f6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.256723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:26.257201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dede7f-b00a-4cbc-ac63-62660472f3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.257672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dede7f-b00a-4cbc-ac63-62660472f3f6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.259491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 475/962] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:26.264443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.264783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.265120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.284832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:26.286090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:26.334104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 488/963] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:26.368071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 476/964] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:26 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:26.613768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:26.640788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:26.652890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:26.719156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:26.734461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8802b76f-dff6-4070-ad65-7f4726e90b3d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:26.770585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88c2958-edea-45bb-a793-f8a455f8bd20 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:26.771922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:26.772840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.773369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.773853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.785824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:26.791367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:26.792356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:26.793262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:26.795958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 489/965] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Oct 4 10:08:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:26.802998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:26.814828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:26.815435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.815711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.815980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.816643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:26.817996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:26.838358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 477/966] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:08:26 2022] POST /identity/v3/services => generated 242 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:26.931113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:26.938623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b5b38-0eda-44e2-b9a7-377321ac4cf8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:26.985594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:26.986666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:26.987164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:26.987638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:26.988548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:26.989062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:26.990701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc31aa-ecab-41fe-8352-71b130e7b748 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.001304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 490/967] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:08:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:27.007493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.010729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e0988-8982-484e-b378-8985d37e8e54 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.010989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e0988-8982-484e-b378-8985d37e8e54 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.011259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e0988-8982-484e-b378-8985d37e8e54 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.012746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 491/968] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:27.018831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.019138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.019373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.041440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:27.042652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:27.128736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 492/969] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:27.151364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9e00ef-38f8-4244-b312-b6b14f2b3d6f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.170319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:27.190422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.191358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.191832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.192319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.204206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:27.206072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:27.221837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:27.223244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:27.224390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:27.224390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:27.224390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:27.225924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 478/970] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:27.231952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.324226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:27.342518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9f8bc-2854-41ac-83c7-948021408207 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.379205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.380139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.380587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.381039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.381855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:27.382373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:27.383919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.396626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 493/971] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:27.402585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.428624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.519862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdaf21a-afa2-49e5-bac6-272f527dcbc9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.539874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f6012b-aa72-4237-92b0-9d79ac3943ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.555872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.556863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.557419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.557841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.566965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:27.568800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.574448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.575076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.575355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.575628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.576211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:27.577548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-906006c9-3e1b-438a-97fb-f73206b9f33d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:27.582087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:27.583249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 04 10:08:27.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:27.586280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 479/972] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:27.587064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:27.588417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:27.589279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:27.590738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9ee62-8ffa-4917-87be-08dd286c64f7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.591637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9ee62-8ffa-4917-87be-08dd286c64f7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.592038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9ee62-8ffa-4917-87be-08dd286c64f7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.593154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 494/973] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:27.594052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.594236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 480/974] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:27.595090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9ee62-8ffa-4917-87be-08dd286c64f7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.608394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:27.641675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:27.703374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7484eba8-69b0-4405-b590-bdbfdab473e3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.722006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9ee62-8ffa-4917-87be-08dd286c64f7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.741081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.741675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.741950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.742241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.742979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:27.743285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:27.744727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209cdc80-9b95-456b-8fd3-813e379a4bb4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.759310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.759913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.760183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.760460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.761078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:27.761376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:27.762759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:27.768954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 495/975] 10.222.0.13 () {56 vars in 1149 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:27.768954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.768954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.768954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.774016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 481/976] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:27.777932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.784261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:27.785904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:27.850892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:27.859490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 496/977] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:27 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:27.860612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:27.898739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:27.910334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf6009-04cb-43e5-92bf-25c9fd7a6084 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:27.946675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:27.947345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:27.947624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:27.947901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:27.977952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221df20c-a97e-4d8b-becf-c0b04844a0ee None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.017908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.019141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.020047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.020807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.021659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:28.022193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.024153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.027113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.028632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.033209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 497/978] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:28.042087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.128707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:28.136383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:28.147786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 482/979] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:27 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:28.148814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.176621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baf3164-8618-4e8f-a662-6f82b8b729d1 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.214578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.215296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.215577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.215846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.216556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] RBAC: Adding query filter params (interface=admin, service_id=18c858fd034d49bb8fed123d3dd98577, region_id=RegionOne) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:28.216862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.218275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.228550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 498/980] 10.222.0.13 () {56 vars in 1249 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/endpoints?service_id=18c858fd034d49bb8fed123d3dd98577&interface=admin®ion_id=RegionOne => generated 193 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:28.229506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.256620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87fee26-52ef-46c2-bcf9-dedd7c9572f0 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.292664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.293224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.293477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.293752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.294415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.295830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.299418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING keystone.server.flask.common [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 04 10:08:28.313311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.sql.core [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 10:08:28.313311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 10:08:28.313311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [parameters: {'user_id': 'b87558bcb73e439d8d4a144fe9c214e5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 10:08:28.313311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81362) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 04 10:08:28.324411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:08:28.325381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application yield resource Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application next(self.gen) Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.rollback() Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.force_reraise() Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise self.value Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application session.commit() Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 04 10:08:28.326829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._prepare_impl() Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.session.flush() Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._flush(objects) Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application compat.raise_( Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application flush_context.execute() Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rec.execute(self) Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 04 10:08:28.328279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 04 10:08:28.329471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result = self._query(query) Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application conn.query(q) Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application result.read() Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 04 10:08:28.330967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [parameters: {'user_id': 'b87558bcb73e439d8d4a144fe9c214e5', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:28.332037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 04 10:08:28.333014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 483/981] 10.222.0.13 () {60 vars in 1131 bytes} [Tue Oct 4 10:08:28 2022] POST /identity/v3/users => generated 160 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:28.334672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.344885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9f4b2-2156-48e7-a6a6-7e7443614797 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.369289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:28.382152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.382967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.383245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.383565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.392244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:28.396219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.398425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:28.412777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:28.413778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:28.415324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:28.415324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:28.415324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:28.415324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 499/982] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:28.415324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.443816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b1ca44-b936-48b9-a6a5-291d172add9a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.480004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.480606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.480883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.481151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.481747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.483338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.493080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 484/983] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:08:28.493936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.544942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01786136-3e69-4bc5-b745-7efe188677c9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.598518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.599577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.600053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.600521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.614018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.615883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.621317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44fe9e6-2e00-4849-b458-870e24c3866f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:28.630671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:28.631674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 04 10:08:28.633162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:28.641967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 500/984] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:28.644121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.658026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:28.677979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.678992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.679488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.679943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.680581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:28.680985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.682248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.686607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:28.695286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 485/985] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:28.700746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:28.769441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd07863-e84e-4474-a2e7-dcadb8f1486f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.805248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.805856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.806404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.806833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.808006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:28.808516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.810765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.830108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9cb37b-be78-42b6-9efc-067e433f593e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:28.854334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 501/986] 10.222.0.13 () {56 vars in 1101 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:28.869192 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:28.870282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:28.870793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:28.871263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:28.879855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:28.881676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:28.895175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:28.896137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:28.897032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:28.899591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 486/987] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:28.905414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:29.013557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:29.024074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe55010-cfb9-4434-a9bc-ef2d6d9936aa None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:29.068577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:29.069547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.070046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.070566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.071507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:29.072017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:29.073673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19243d6c-3c63-4bb5-8650-bd06bb1de77a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:29.087637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 502/988] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:29.094536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:29.145417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:29.161375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb128e1-88aa-49cc-aa41-06aad9880d95 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.161750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb128e1-88aa-49cc-aa41-06aad9880d95 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.161930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb128e1-88aa-49cc-aa41-06aad9880d95 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.163565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 503/989] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:29.168429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.168719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.168996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.189254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:29.190456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:29.214551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0f475-9ea7-4636-b742-a84f65d55ac8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:29.252654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:29.253382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.253670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.254053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.254815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] RBAC: Adding query filter params (role.id=b98f044251bf4fb4ab172eb56bedeb2f, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=9f385f2a341a4162af72414f4dc0ee91) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:29.255125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:29.256541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:29.261787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 504/990] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:29 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:29.270234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 487/991] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:29 2022] GET /identity/v3/role_assignments?user.id=9f385f2a341a4162af72414f4dc0ee91&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=b98f044251bf4fb4ab172eb56bedeb2f => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:29.529816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:29.614963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:29.638981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f9655d-cffa-4c22-9071-d983094cf9c6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:29.673651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:29.674305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.674582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.674848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.675426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:29.676782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:29.685283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:29.686559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:29.688092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:29.690266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 505/992] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:29 2022] GET /identity/v3/services/compute => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:29.692936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:29.798319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5b892f-71be-4dc0-a464-e10df4300461 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:29.801418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:29.833430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:29.834026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.834332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.834604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.835221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] RBAC: Adding query filter params (name=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:29.835518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:29.836859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:29.846268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 488/993] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:29 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:29.847468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:29.946963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4c025a-ae32-44bf-b6cb-ecfc131c9574 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:29.980406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:29.981003 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:29.981274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:29.981541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:29.982240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:29.982546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:29.984056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:29.993536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 506/994] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:29 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:29.995141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:30.097244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8abc9e-cafa-42ec-ac06-a3b15b5995ea None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:30.130803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:30.131409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:30.131684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:30.131956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:30.132645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] RBAC: Adding query filter params (interface=public, service_id=1620edba60624d11ac0697669209c1cf, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:30.132949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:30.134329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f281ec1-a2c8-4fa3-83ec-6b3d298ba430 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:30.144429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 489/995] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:29 2022] GET /identity/v3/endpoints?service_id=1620edba60624d11ac0697669209c1cf&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:30.145585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:30.161471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:30.251930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4be37-d745-4fbd-9b5c-1502c99e529e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:30.285599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc8d90-3115-448f-88d4-53c453d1e3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:30.286216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc8d90-3115-448f-88d4-53c453d1e3de None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:30.286503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc8d90-3115-448f-88d4-53c453d1e3de None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:30.286771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc8d90-3115-448f-88d4-53c453d1e3de None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:30.287369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc8d90-3115-448f-88d4-53c453d1e3de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:30.288726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc8d90-3115-448f-88d4-53c453d1e3de None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:30.295117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 507/996] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:30 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:08:30.731187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8184dc-0d95-4d67-b46b-cfea9dc887d9 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:30.731714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8184dc-0d95-4d67-b46b-cfea9dc887d9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:30.731714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8184dc-0d95-4d67-b46b-cfea9dc887d9 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:30.733093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 490/997] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:30.738885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:30.739255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:30.739413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:30.757664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:30.758858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:30.826257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 508/998] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:30 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:31.097218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8184dc-0d95-4d67-b46b-cfea9dc887d9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:31.128111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:31.202443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8184dc-0d95-4d67-b46b-cfea9dc887d9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:31.236789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:31.237419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:31.237695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:31.237965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:31.238573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:31.239939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:31.248617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:31.250661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 04 10:08:31.251528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:31.254288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 491/999] 10.222.0.13 () {56 vars in 1126 bytes} [Tue Oct 4 10:08:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:31.256623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:31.264958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:31.357742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172fdae8-b528-4f90-901e-c61d8c247c7a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:31.396643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:31.397315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:31.397594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:31.397860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:31.398516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:31.398814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:31.400217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f04dfe2-d1ca-4d5c-b666-f8c2f0f995fb None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:31.409523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 509/1000] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:31 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:31.411213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:31.479820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb2895-7545-4ebf-aee1-b2b255d07134 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:31.480133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb2895-7545-4ebf-aee1-b2b255d07134 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:31.480367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabb2895-7545-4ebf-aee1-b2b255d07134 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:31.481848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 510/1001] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:31.486995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:31.487283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:31.487558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:31.506872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:31.508049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:31.514170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:31.521120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370754-fc7f-4d8e-b9c0-dd2eb163b4f7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:31.557532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:31.558600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:31.559085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:31.559576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:31.560311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:31.560625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:31.562604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:31.567592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 492/1002] 10.222.0.13 () {56 vars in 1122 bytes} [Tue Oct 4 10:08:31 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:31.572963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:31.578415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:31.583384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 511/1003] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:31 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:31.682054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e110156-3ab9-467d-99e6-61ec4cfe7ed8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:31.716153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:31.716749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:31.717024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:31.717290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:31.717935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:31.719577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:31.738784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 493/1004] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:31 2022] POST /identity/v3/endpoints => generated 325 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:31.857606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:31.967338 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74aa523-2d72-4a51-b601-20c007483934 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:32.011321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:32.011955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:32.012236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:32.012508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:32.013089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:32.014472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:32.024087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:32.025071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:32.025981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 512/1005] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Oct 4 10:08:31 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:32.031180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:32.054265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:32.141611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a879f0-01e8-4363-8238-c0a6c4c6e032 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:32.174754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:32.175605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:32.176005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:32.176394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:32.177233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] RBAC: Adding query filter params (name=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:32.177763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:32.179539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889484c-d15e-4ce6-b6bf-0e834e0eb4fb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:32.184772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 494/1006] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:32 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:32.190959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:32.252021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:32.293568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e517384-e542-4bcc-b548-fe483c995ef3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:32.328221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:32.328830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:32.329111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:32.329379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:32.330013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] RBAC: Adding query filter params (type=image) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:32.330347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:32.331761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ca514-04bd-48fc-b635-89ef23c0d1a6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:32.336596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 513/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:32 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:32.471191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ccae-7c14-4470-af5a-321b344b271d None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:32.471544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ccae-7c14-4470-af5a-321b344b271d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:32.471754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4851ccae-7c14-4470-af5a-321b344b271d None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:32.473340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 495/1008] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:32.479038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:32.479344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:32.479594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:32.498682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:32.500477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:32.573279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 514/1009] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:32 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:32.851087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ccae-7c14-4470-af5a-321b344b271d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:32.934354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:32.952725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4851ccae-7c14-4470-af5a-321b344b271d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:32.986108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:32.986755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:32.987029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:32.987298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:32.987866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:32.989215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:32.997780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:32.998992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 04 10:08:33.000677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:33.002480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 496/1010] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:33.005330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:33.070926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:33.120236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21826fa9-4c98-453b-9de5-1762e3a9fc16 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:33.153032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:33.153642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:33.153915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:33.154183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:33.154823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:33.155121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:33.156995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:33.166292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 515/1011] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:33 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:33.167517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:33.271122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5f7167-192b-4108-ab62-e939bf010d06 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:33.303179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:33.303772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:33.304045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:33.304310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:33.304919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:33.305214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:33.306588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e9f3a-48e3-4e0a-a544-734d206f616c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:33.311403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 497/1012] 10.222.0.13 () {56 vars in 1112 bytes} [Tue Oct 4 10:08:33 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:33.317335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:33.423645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44b23dc-0192-41eb-9c9a-5cae0fb00e51 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:33.440015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:33.458365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd0ab3-f97a-45a5-ac2e-34c6369c9532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:33.458959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd0ab3-f97a-45a5-ac2e-34c6369c9532 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:33.459232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd0ab3-f97a-45a5-ac2e-34c6369c9532 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:33.459498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd0ab3-f97a-45a5-ac2e-34c6369c9532 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:33.460156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd0ab3-f97a-45a5-ac2e-34c6369c9532 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:33.461550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd0ab3-f97a-45a5-ac2e-34c6369c9532 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:33.484841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 516/1013] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:33 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:34.037404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4cb2c-3d83-4876-a28a-5c5fbcc40a10 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:34.037943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4cb2c-3d83-4876-a28a-5c5fbcc40a10 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:34.037943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e4cb2c-3d83-4876-a28a-5c5fbcc40a10 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:34.039363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 498/1014] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:34.044701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7a28c-6bce-4d92-a2df-41223b5055e4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:34.045101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7a28c-6bce-4d92-a2df-41223b5055e4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:34.045301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad7a28c-6bce-4d92-a2df-41223b5055e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:34.065806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4ad7a28c-6bce-4d92-a2df-41223b5055e4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:34.067028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4ad7a28c-6bce-4d92-a2df-41223b5055e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:34.141398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 517/1015] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:34 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:34.414034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4cb2c-3d83-4876-a28a-5c5fbcc40a10 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:34.431322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:34.522980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e4cb2c-3d83-4876-a28a-5c5fbcc40a10 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:34.530687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073fe00-804d-4481-895f-0f47d6104448 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:34.530922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073fe00-804d-4481-895f-0f47d6104448 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:34.531190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e073fe00-804d-4481-895f-0f47d6104448 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:34.532544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 518/1016] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:34.537372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:34.537655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:34.537929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:34.555253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:34.556401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:34.556558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:34.557026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:34.557299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:34.557567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:34.558691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:34.560088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:34.562236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 499/1017] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:34.568022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:34.637193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 519/1018] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:34 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:34.654155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:34.675193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23188a49-aa54-4c5b-9f7d-c545739a026c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:34.708291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f994ad-b9d4-4c2d-ae7b-6c2e39c5e4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:34.708891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f994ad-b9d4-4c2d-ae7b-6c2e39c5e4be None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:34.709164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f994ad-b9d4-4c2d-ae7b-6c2e39c5e4be None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:34.709483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f994ad-b9d4-4c2d-ae7b-6c2e39c5e4be None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:34.710121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f994ad-b9d4-4c2d-ae7b-6c2e39c5e4be None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:34.711556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f994ad-b9d4-4c2d-ae7b-6c2e39c5e4be None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:34.741589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 500/1019] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:08:34 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:34.908137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:34.939299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:35.012008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34175fb-2bc5-4f9a-85b8-6d6cc1209640 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:35.047317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d1965c-1094-42b0-b36a-a9eadc78167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:35.047918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1965c-1094-42b0-b36a-a9eadc78167d None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:35.048182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1965c-1094-42b0-b36a-a9eadc78167d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:35.048447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d1965c-1094-42b0-b36a-a9eadc78167d None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:35.049096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1965c-1094-42b0-b36a-a9eadc78167d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:35.050556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d1965c-1094-42b0-b36a-a9eadc78167d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:35.066419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 520/1020] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:08:34 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:35.760922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab3717-d352-45d7-a509-7408485c1412 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:35.761536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab3717-d352-45d7-a509-7408485c1412 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:35.761536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ab3717-d352-45d7-a509-7408485c1412 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:35.762986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 501/1021] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:35.768317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:35.768606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:35.768880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:35.786904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:35.788081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:35.862180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 521/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:35 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:36.127689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab3717-d352-45d7-a509-7408485c1412 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:36.234843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ab3717-d352-45d7-a509-7408485c1412 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:36.262629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:36.268519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:36.269197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:36.269819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:36.270467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:36.281481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:36.286152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:36.287229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:36.288088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:36.290266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 502/1023] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Oct 4 10:08:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:36.293479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:36.311037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:36.395747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3efcb2-0f50-46a9-aab1-6502f25aa7c9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:36.429234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:36.429845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:36.430103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:36.430400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:36.431050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:36.431345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:36.432765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:36.445807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 522/1024] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:08:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:36.447262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:36.543646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84a40b8-adf7-424c-92ee-4fe482ddf5fe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:36.577272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:36.577892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:36.578157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:36.578450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:36.588591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:36.590012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:36.604222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:36.605190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:36.606064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:36.606064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:36.606064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:36.606064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 503/1025] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:36.612145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:36.704448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:36.729615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58765195-6ca9-468a-afb3-92a0a348c469 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:36.764772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:36.765422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:36.765700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:36.766006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:36.766632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:36.766921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:36.768272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:36.785551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 523/1026] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:36.787026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:36.838729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:36.903882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ab29d7-53b4-465d-8180-95844dd51cf3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:36.939732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:36.940356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:36.940630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:36.940895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:36.948667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:36.950096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-dee5f2b4-be57-4107-8c5b-f309f1e9db18 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:36.961381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:36.962471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:36.963696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:36.967448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 504/1027] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:08:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:36.969503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.007891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128fe93-905e-4d63-9f55-c9a0c9febadb None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.008167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128fe93-905e-4d63-9f55-c9a0c9febadb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.008437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a128fe93-905e-4d63-9f55-c9a0c9febadb None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.009838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 505/1028] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:37.015239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.015690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.015968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.036963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:37.039392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:37.095712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69393b6-d643-4374-862a-17181dbb365e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.142247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.142247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.142247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.142247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.142650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:37.142650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.142778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 506/1029] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:37.146084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5928d02-4537-4880-8c7b-16d99aa6c17c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.152547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.164073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 524/1030] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:08:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:37.170015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.207513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.274127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd6bf48-90d5-46fc-be69-086b41ade3fd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.285357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c41d5-41fa-4468-8440-9e6bf0e3a8a3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.285651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c41d5-41fa-4468-8440-9e6bf0e3a8a3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.285928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3c41d5-41fa-4468-8440-9e6bf0e3a8a3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.287318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 525/1031] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:37.292387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6c408-a682-497b-891b-a4d843607667 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.292766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6c408-a682-497b-891b-a4d843607667 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.293048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df6c408-a682-497b-891b-a4d843607667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.312635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0df6c408-a682-497b-891b-a4d843607667 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:37.313815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0df6c408-a682-497b-891b-a4d843607667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:37.314681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.315294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.315563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.315838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.386210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.387680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.388166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 526/1032] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:37 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:37.414736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6c408-a682-497b-891b-a4d843607667 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.426338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.499618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 507/1033] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:37.505389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.524713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df6c408-a682-497b-891b-a4d843607667 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.552569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.562601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.563223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.563498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.563764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.564850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.566379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.570250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 527/1034] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:37.573728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.614138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa2f434-c13d-4be4-9b7a-8f860f25fa13 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.649736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.650516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.650854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.651193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.653080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.663586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.665027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:37.679786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:37.680735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:37.681651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:37.681651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:37.681651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:37.681651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 508/1035] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:37.681885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.691769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b553015d-6061-4a24-8c72-04118c2fc296 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.725316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.725903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.726182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.726482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.727130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.728549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.750574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 528/1036] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:08:37 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:37.751478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.783466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663bfa84-353e-4c26-8662-dc94e91f0f7d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.822268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.822975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.823189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.823470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] PATH_INFO: `/v3/services/image` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.824074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.825469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.828180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:37.835393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:37.836454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:37.837292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 509/1037] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/services/image => generated 86 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:37.844157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:37.900111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5e27d-5ad1-45bf-a750-db11a8223643 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.904354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:37.942175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616affe-9837-4c73-822b-38102146a55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:37.943245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616affe-9837-4c73-822b-38102146a55e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:37.943520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616affe-9837-4c73-822b-38102146a55e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:37.944439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616affe-9837-4c73-822b-38102146a55e None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:37.944726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616affe-9837-4c73-822b-38102146a55e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:37.945594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616affe-9837-4c73-822b-38102146a55e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:37.947240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616affe-9837-4c73-822b-38102146a55e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:37.959994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 529/1038] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:37.963212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e9bd7c-6bcc-43d7-abce-60621535accc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:37.966637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616affe-9837-4c73-822b-38102146a55e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:38.004688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.005292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.005569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.005840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.006603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] RBAC: Adding query filter params (name=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:38.006906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.008289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.013577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 510/1039] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:38.019907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:38.078793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616affe-9837-4c73-822b-38102146a55e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:38.091394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:38.118242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.119024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.119302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.119886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.128126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c149bf-bace-4a4b-bae7-d34b80554ea8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:38.128598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.130223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.142791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:38.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:38.146228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:38.147556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 530/1040] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:38.152538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:38.169581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.170182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.170501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.170793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.171419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] RBAC: Adding query filter params (type=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:38.171772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.173177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.184466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 511/1041] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:38 2022] GET /identity/v3/services?type=image => generated 350 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:38.185618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:38.262375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec324d8-676a-41ae-9545-1cab24e774d4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:38.295951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9cac6b-715e-4774-98c8-daafdff1da5d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:38.299510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.300136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.300478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.300759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.301523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:38.301835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.303723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.317786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 531/1042] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:38.323301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:38.336586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.337134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.337418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.337695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.338691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] RBAC: Adding query filter params (interface=public, service_id=4690f14fec0b43468b7ce5605ee5eca1, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:38.338999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.340357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.348499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 512/1043] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:08:38 2022] GET /identity/v3/endpoints?service_id=4690f14fec0b43468b7ce5605ee5eca1&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:38.354280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:38.360130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:38.402636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:38.437275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8512ca9-accc-484f-be56-5050725208aa None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:38.461643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6c9ead-9e29-4027-9b54-1cbd5a7e5fa8 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:38.472389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.472990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.473264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.473697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.474257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] RBAC: Adding query filter params (role.id=b98f044251bf4fb4ab172eb56bedeb2f, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=a8514e96615c4785aefde8261120afe3) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:38.474568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.475983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5994331-5e50-49fe-a778-a1e803ffc226 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.481610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 532/1044] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:38 2022] GET /identity/v3/role_assignments?user.id=a8514e96615c4785aefde8261120afe3&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=b98f044251bf4fb4ab172eb56bedeb2f => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:38.498746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883b05b-b798-4b00-b23a-b43cf7c2f262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:38.499408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b05b-b798-4b00-b23a-b43cf7c2f262 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:38.499680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b05b-b798-4b00-b23a-b43cf7c2f262 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:38.500160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883b05b-b798-4b00-b23a-b43cf7c2f262 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:38.500789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883b05b-b798-4b00-b23a-b43cf7c2f262 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:38.502182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883b05b-b798-4b00-b23a-b43cf7c2f262 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:38.509732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 513/1045] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:08:38 2022] GET /identity/v3/services => generated 1957 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:08:40.023718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7983-9ef7-45fe-9cf1-2530aa250fc1 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.024218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7983-9ef7-45fe-9cf1-2530aa250fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.024218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bec7983-9ef7-45fe-9cf1-2530aa250fc1 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.025687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 533/1046] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:40.031986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.032391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.032554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.051632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:40.052818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:40.123610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 514/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:40.394141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7983-9ef7-45fe-9cf1-2530aa250fc1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:40.498570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:40.507938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bec7983-9ef7-45fe-9cf1-2530aa250fc1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:40.543217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db39e21-8d5a-4e28-804f-d84c145c8418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:40.543832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39e21-8d5a-4e28-804f-d84c145c8418 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.544103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39e21-8d5a-4e28-804f-d84c145c8418 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.544369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db39e21-8d5a-4e28-804f-d84c145c8418 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.555737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db39e21-8d5a-4e28-804f-d84c145c8418 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-2db39e21-8d5a-4e28-804f-d84c145c8418 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:40.560724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:40.562409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:40.564859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 534/1048] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:40.568509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:40.607621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:40.683920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d0d325-158e-47e1-b6f4-fb00d666a507 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:40.690020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef63a4a-a87c-4f7c-8bfe-25380a6335f8 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.690336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef63a4a-a87c-4f7c-8bfe-25380a6335f8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.690611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef63a4a-a87c-4f7c-8bfe-25380a6335f8 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.692036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 535/1049] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:40.693671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94f58-0b81-4a47-a1cd-450030b038cf None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.693909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94f58-0b81-4a47-a1cd-450030b038cf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.694187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe94f58-0b81-4a47-a1cd-450030b038cf None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.695438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 536/1050] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:40.697084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.697364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.697638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.715556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:40.716731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:40.720446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:40.721052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.721325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.721593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.722265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:40.722564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:40.723903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6b735f-b375-47ac-8ace-d8083f5e66d5 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:40.731771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 515/1051] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:40.732892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.733214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.734243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.751236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:40.752418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:40.784123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 537/1052] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:40.784976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:40.830013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 516/1053] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:40 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:40.874463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:40.891189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852e8b4a-44ef-4f8a-9ca7-8ef9edc620d5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:40.925067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:40.925687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:40.925960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:40.926247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] PATH_INFO: `/v3/users/65352c83894e4dd3969ce0ba885ae520` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:40.939404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] RBAC: Authorizing `identity:get_user(user_id=65352c83894e4dd3969ce0ba885ae520)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:40.940784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:40.942389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 538/1054] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:40 2022] GET /identity/v3/users/65352c83894e4dd3969ce0ba885ae520 => generated 285 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:40.948478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:40.977554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.052820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bed5454-31f1-4456-9f48-cc3fda85f696 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.060789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.090407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.091014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.091285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.091557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.159583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.161005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfd71c4-cde2-4f72-a5ac-190247119ec1 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.161109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.187773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.194811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.195378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.195652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.195920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.207038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:41.211507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:41.212553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:41.213638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 539/1055] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:41.214344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.216906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.262358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 517/1056] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:40 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:41.263199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.316576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14517b67-9db0-4a63-b555-50bd63b2a090 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.350575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.351191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.351467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.351735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] PATH_INFO: `/v3/services/image` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.352312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.353665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.360498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f12c390-7f04-4bc7-a208-3db5821cf34c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:41.362143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:41.363150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:41.363989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 540/1057] 10.222.0.13 () {56 vars in 1105 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:41.364768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.390466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.399267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.399875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.400147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.400413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.401063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:41.401358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.402738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.410701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 518/1058] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:41.411479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.432541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.467991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3bbeb-b398-49a5-a43e-56263e230f0e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.501674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.502323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.502605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.502876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.503985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] RBAC: Authorizing `identity:get_project(project_id=54a1fb95d53148ba92c040d07ca973f7)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.505404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.507072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 541/1059] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:41.507827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.508347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6449ecd9-4548-4564-acf0-7cc35958a0d7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.543930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.544528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.544805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.545077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.546325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] RBAC: Adding query filter params (name=image) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:41.546631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.547982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.552733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 519/1060] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:41.553501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.606035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d0345-21b9-48e6-8831-4385babf3be7 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.607655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.642383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.643010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.643280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.643546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.645396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.655023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.656464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d54fe8-e2d2-449b-b093-93d3c77301bb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.656580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:41.670229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:41.671935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:41.673096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:41.673096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:41.673096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:41.673096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 542/1061] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:41.673096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.694712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.695321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.695600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.695891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.697078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=54a1fb95d53148ba92c040d07ca973f7, user.id=65352c83894e4dd3969ce0ba885ae520) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:41.697398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.698833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.704803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 520/1062] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/role_assignments?user.id=65352c83894e4dd3969ce0ba885ae520&scope.project.id=54a1fb95d53148ba92c040d07ca973f7&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:41.705586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.792982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1471be03-3684-4f96-bd66-6dda86ae1623 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.823195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221a0027-0c62-4187-b1ed-ccae7a759daf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.832354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.833241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.833840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.834108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.834793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] RBAC: Adding query filter params (type=image) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:41.835102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.836606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.842301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.844467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 543/1063] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:41.851178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.862969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.863731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.864009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.864281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:41.864929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:41.865228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:41.866630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:41.881016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 521/1064] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:41.886623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:41.908066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:41.960968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a15113-48ef-4911-bd5b-683bbd1fbc46 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:41.997914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:41.998560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:41.999364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:41.999644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.000442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.001799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.003187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb01550-c646-47c7-85b1-5e930ba01790 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:42.024159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 544/1065] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:08:41 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:08:42.051463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:42.052421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:42.052889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:42.053941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] PATH_INFO: `/v3/users/nova` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.063544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.065476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:42.078730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:42.079836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:42.080772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:42.086637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 522/1066] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:08:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:42.087081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:42.205264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77acbea9-6913-46ed-9050-309ec756c440 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:42.214366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:42.246527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:42.247140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:42.247416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:42.247684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.248403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:42.248856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.250433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.269048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 545/1067] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:08:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:42.275098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:42.311540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:42.378951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c1eef5-9a27-4308-b98d-52c6472be69b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:42.413579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:42.414229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:42.414514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:42.414788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.425698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.427179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:42.442888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:42.443926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:42.444883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:42.444883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:42.444883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:42.446296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 523/1068] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:42.449982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:42.554881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8649b9d0-e5e2-4a3a-b003-f4f1a9fa4963 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:42.592126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:42.593962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:42.594505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:42.594996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.595895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:42.596406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.598001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.615286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 546/1069] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:42.616638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:42.711352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:42.729215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f591bf-6abe-4445-85e8-eb5c0f55ad57 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:42.766965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:42.767611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:42.767897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:42.768357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.775846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.777272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:42.789698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:42.790680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:42.791571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 524/1070] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:42.797377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:42.822614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:42.897345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c5536-92f7-4fee-a8ce-86d981e83366 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:42.930499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:42.931119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:42.931391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:42.931658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:42.932366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:42.932665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:42.934131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917b098-5735-4716-95bf-a59d19ca63df None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:42.953634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 547/1071] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:42.957431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:43.056865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c414f70-18f8-4fe4-ac98-e3aff3ba1393 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:43.094428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c1b397-2a53-4707-bb7c-d261fb9be4cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:43.095296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b397-2a53-4707-bb7c-d261fb9be4cb None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:43.095599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b397-2a53-4707-bb7c-d261fb9be4cb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:43.095837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c1b397-2a53-4707-bb7c-d261fb9be4cb None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/a8514e96615c4785aefde8261120afe3/roles/b98f044251bf4fb4ab172eb56bedeb2f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:43.097917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1b397-2a53-4707-bb7c-d261fb9be4cb None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=a8514e96615c4785aefde8261120afe3, role_id=b98f044251bf4fb4ab172eb56bedeb2f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:43.099358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c1b397-2a53-4707-bb7c-d261fb9be4cb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:43.122239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 525/1072] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:42 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/a8514e96615c4785aefde8261120afe3/roles/b98f044251bf4fb4ab172eb56bedeb2f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:08:43.880023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2ecf4-8bd1-4632-9d55-3e5876f3a056 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:43.880533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2ecf4-8bd1-4632-9d55-3e5876f3a056 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:43.880533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2ecf4-8bd1-4632-9d55-3e5876f3a056 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:43.882095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 548/1073] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:43.887085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811b6cb-42b2-4b85-9520-c4a13c2c6b60 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:43.887374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811b6cb-42b2-4b85-9520-c4a13c2c6b60 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:43.887645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7811b6cb-42b2-4b85-9520-c4a13c2c6b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:43.901009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:43.907435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7811b6cb-42b2-4b85-9520-c4a13c2c6b60 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:43.909435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7811b6cb-42b2-4b85-9520-c4a13c2c6b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:43.975751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 526/1074] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:44.223185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50e86-7358-43ee-8410-9d39b9a198c1 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.223365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50e86-7358-43ee-8410-9d39b9a198c1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.223635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f50e86-7358-43ee-8410-9d39b9a198c1 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.224983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 549/1075] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:44.230387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.230836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.230998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.245139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f50e86-7358-43ee-8410-9d39b9a198c1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.250148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:44.251363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:44.317870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 527/1076] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:44.338546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:44.357146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f50e86-7358-43ee-8410-9d39b9a198c1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:44.392287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:44.392904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.393179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.393447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.404030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:44.408878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:44.409806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:44.410724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:44.414596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 550/1077] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:44.415925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.481895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:44.517730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9144aa9-b979-4f00-af97-85e5fdf0b442 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:44.551339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:44.551949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.552233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.552505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.553158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:44.553461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:44.554852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:44.568276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 528/1078] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:44.569422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.596143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.684787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f406ba8d-2e71-46c9-91b6-74108c600513 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:44.706394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfd0eca-1251-41f0-b236-ad2ade8dc18e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:44.720221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:44.720850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.721130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.721400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] PATH_INFO: `/v3/users/65352c83894e4dd3969ce0ba885ae520` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.722541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] RBAC: Authorizing `identity:get_user(user_id=65352c83894e4dd3969ce0ba885ae520)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:44.723969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:44.730243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 551/1079] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:44 2022] GET /identity/v3/users/65352c83894e4dd3969ce0ba885ae520 => generated 285 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:44.731196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.737813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:44.746223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:44.746802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.747078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.747349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.748151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:44.820960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:44.822414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:44.841650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd60ea4c-5f95-46ca-88a1-c232ef1b59dc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:44.876648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:44.877279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:44.877566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:44.877834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:44.878984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] RBAC: Authorizing `identity:get_project(project_id=54a1fb95d53148ba92c040d07ca973f7)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:44.880411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:44.882428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 552/1080] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:08:44 2022] GET /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:44.887855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.938598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 529/1081] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 10:08:44.940941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:44.968825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:44.975347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:44.998951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a667eabe-0625-4f3e-9788-3c903b99521d None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:45.036493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:45.037177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.037455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.037728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7/users/65352c83894e4dd3969ce0ba885ae520/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.039928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] RBAC: Authorizing `identity:create_grant(project_id=54a1fb95d53148ba92c040d07ca973f7, user_id=65352c83894e4dd3969ce0ba885ae520, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:45.041291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:45.044766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e4df8e-3521-4192-b4b9-e9c4d96f0e0a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:45.056460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 553/1082] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:44 2022] PUT /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7/users/65352c83894e4dd3969ce0ba885ae520/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:08:45.084915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:45.085912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.086397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.086842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.100039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:45.101803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-a1fc87ed-447f-463f-b6c5-83f700503957 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:45.119504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:45.120514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:45.121438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:45.121438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:45.121438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:45.123103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 530/1083] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:45.129217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:45.250676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b4cf13-f1de-415d-b798-c25a5fb7ee46 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:45.284625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:45.285239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.285513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.285809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.286496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:45.286799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:45.288166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:45.290611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef373e-e64f-42ce-ace0-8e6d72d027a4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.291071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef373e-e64f-42ce-ace0-8e6d72d027a4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.291183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eef373e-e64f-42ce-ace0-8e6d72d027a4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.292466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 531/1084] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:45.298529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.298807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.299078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.302028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:45.306710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 554/1085] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:45.322254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:45.323750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:45.400337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 532/1086] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:45.671397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:45.771578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146dc44-a591-4964-8db9-ba32e5e7f677 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:45.804313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:45.804931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.805208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.805479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.816013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:45.820785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:45.821722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:45.822740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 555/1087] 10.222.0.13 () {56 vars in 1120 bytes} [Tue Oct 4 10:08:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:45.828582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:45.840209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:45.924567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccaf1ab-2d95-4a80-b0c6-1504c24bef03 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:45.958072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:45.958732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:45.959012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:45.959285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:45.959960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:45.960263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:45.961669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:45.971059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 533/1088] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:08:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:45.976853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:46.039475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:46.079346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266a297-4ade-44d1-b4e8-6c37da3c561f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:46.111881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:46.112489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:46.112759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:46.113023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:46.123337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:46.124864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:46.138676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:46.140368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:46.141723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:46.141723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:46.141723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:46.141723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 556/1089] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:46.145991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:46.215501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:46.246139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94023467-5ddb-4bff-b505-74e7cd6a9938 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:46.278765 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:46.279357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:46.279629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:46.280554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:46.281153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:46.281443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:46.282822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:46.294083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 534/1090] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:46.300128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:46.400486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddc2a0-2ca8-4785-9fd2-d0df466a6306 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:46.413513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:46.434248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:46.434878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:46.435171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:46.435426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:46.442825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:46.444251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:08:46.455734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:46.456829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:46.457846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 557/1091] 10.222.0.13 () {56 vars in 1093 bytes} [Tue Oct 4 10:08:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:46.462721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:46.559635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbbb678-e06c-4eeb-96d1-e2a3a46407a4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:46.590015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:46.596493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:46.597085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:46.597359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:46.597626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:46.598378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:46.598675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:46.600132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:46.617296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 535/1092] 10.222.0.13 () {56 vars in 1133 bytes} [Tue Oct 4 10:08:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:46.622972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:46.721198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c521b-e9bf-4309-9418-9bff190f318e None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:46.754390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:46.754989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:46.755261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:46.755525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:46.794413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:46.820544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:46.821918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:46.917538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 558/1093] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:46.923390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.019672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf86ef25-ddde-487e-b825-48a798cf14bc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.052034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.052651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.052924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.053189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.062817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:47.064231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:08:47.078852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:47.079948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:08:47.080991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:08:47.080991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:08:47.080991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:47.080991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 536/1094] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:47.085605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.157243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:47.187250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d565c-56cd-43c1-9774-de2bcb2f3b03 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.221774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.222415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.222699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.222974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.223616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:47.223921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:47.225290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3962e0f-0bee-4fb6-bc40-b23cda0f0a64 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:47.236390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 559/1095] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:47.242034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.245888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a1e34-5cc2-47d4-b9d4-ebb5765e109e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.246176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a1e34-5cc2-47d4-b9d4-ebb5765e109e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.246481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0a1e34-5cc2-47d4-b9d4-ebb5765e109e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.247763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 560/1096] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:08:47.253293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.253934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.254234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.273161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:47.274941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:47.286469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:47.341021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 561/1097] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:08:47.342087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b267444-9f5a-4579-adfb-71cac78d42f7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.377528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.378152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.378459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.378735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.385104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:47.386555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:08:47.397706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:47.398789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:08:47.399690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 537/1098] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:08:47.405566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.465298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:47.503849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a116209-1a10-4994-8e95-7e817e75944d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.536721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.537321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.537599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.537872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.538603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:47.538908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:47.540334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:47.551605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 562/1099] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:47.557786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.610654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.662537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ea2e4b-9ca0-49b4-8a52-ca82e2d311e3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.664116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:47.697357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.697952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.698250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.698525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.699245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] RBAC: Adding query filter params (role.id=b98f044251bf4fb4ab172eb56bedeb2f, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=a8514e96615c4785aefde8261120afe3) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:47.699545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:47.700980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:47.706641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 538/1100] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/role_assignments?user.id=a8514e96615c4785aefde8261120afe3&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=b98f044251bf4fb4ab172eb56bedeb2f => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:47.710869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f6c3d8-4b37-415b-b298-c56b97aad99c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.736306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:47.750932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.751569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.751843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.752109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.764776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:08:47.769710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:08:47.771062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:08:47.772198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:08:47.774563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 563/1101] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:08:47.777048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:47.884715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc7bf8-33b7-459d-b159-5ae292d646dc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:47.918034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:47.918673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:47.918953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:47.919235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:47.919893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:47.920195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:47.922400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:47.931787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 539/1102] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:47.938585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:48.038177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf8abfd-74c3-42d4-9872-2243c42dd8b2 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:48.074887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:48.075522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:48.075800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:48.076069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] PATH_INFO: `/v3/users/65352c83894e4dd3969ce0ba885ae520` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:48.077984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] RBAC: Authorizing `identity:get_user(user_id=65352c83894e4dd3969ce0ba885ae520)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:48.079457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:48.081059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 564/1103] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:47 2022] GET /identity/v3/users/65352c83894e4dd3969ce0ba885ae520 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:48.087077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:48.115522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:48.189761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a812eb2-1c12-4452-baf7-f1a72e1c8acb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:48.225762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:48.226470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:48.226745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:48.227011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:48.293628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:48.295108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:48.322499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:48.408452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 540/1104] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:48 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 10:08:48.414316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:48.467892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:48.519413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e712c46-3bde-44c5-a33f-df0eef6c3cb0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:48.555508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c17d8e-cd0f-4500-9ed9-1e7efea1a0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:48.556132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c17d8e-cd0f-4500-9ed9-1e7efea1a0e5 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:48.556406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c17d8e-cd0f-4500-9ed9-1e7efea1a0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:48.556680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c17d8e-cd0f-4500-9ed9-1e7efea1a0e5 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:48.557777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c17d8e-cd0f-4500-9ed9-1e7efea1a0e5 None admin] RBAC: Authorizing `identity:get_project(project_id=54a1fb95d53148ba92c040d07ca973f7)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:48.559240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c17d8e-cd0f-4500-9ed9-1e7efea1a0e5 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:48.561200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 565/1105] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:08:48 2022] GET /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:48.567455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:48.662974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d25720-e147-4f89-8c0a-0b57a9f63abd None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:48.688662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:48.696722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:48.697323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:48.697600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:48.697871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:48.698622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=54a1fb95d53148ba92c040d07ca973f7, user.id=65352c83894e4dd3969ce0ba885ae520) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:48.698927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:48.700337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854b59bd-ea76-483a-91d1-e8129814e3dd None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:48.706228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 541/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:48 2022] GET /identity/v3/role_assignments?user.id=65352c83894e4dd3969ce0ba885ae520&scope.project.id=54a1fb95d53148ba92c040d07ca973f7&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:08:50.864641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4c68b-4b6b-4043-bcff-b08b7455c595 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:50.865215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4c68b-4b6b-4043-bcff-b08b7455c595 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:50.865215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4c68b-4b6b-4043-bcff-b08b7455c595 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:50.866572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 566/1107] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:50.871968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:50.872373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:50.872510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:50.890766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:50.891863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:50.955154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 542/1108] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:51.227418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4c68b-4b6b-4043-bcff-b08b7455c595 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:51.324723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4c68b-4b6b-4043-bcff-b08b7455c595 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:51.341268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:51.358674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec339d-ae55-4e25-9146-2f0d74008e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:51.359281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec339d-ae55-4e25-9146-2f0d74008e9f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:51.359557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec339d-ae55-4e25-9146-2f0d74008e9f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:51.359822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec339d-ae55-4e25-9146-2f0d74008e9f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:51.360920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec339d-ae55-4e25-9146-2f0d74008e9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:51.362374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec339d-ae55-4e25-9146-2f0d74008e9f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:51.363912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 567/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:51.369604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:51.465025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee69b7e-72b8-42d3-b27d-2285573104c7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:51.486960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:51.498288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b6493-7d1e-4928-8879-94999aef68a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:51.498962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6493-7d1e-4928-8879-94999aef68a8 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:51.499241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6493-7d1e-4928-8879-94999aef68a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:51.499514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6493-7d1e-4928-8879-94999aef68a8 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:51.500162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6493-7d1e-4928-8879-94999aef68a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:51.501564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6493-7d1e-4928-8879-94999aef68a8 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:51.521915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 543/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:08:51 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:08:53.709868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6247-c7c2-489d-a429-ce466a39b5c0 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:53.710398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6247-c7c2-489d-a429-ce466a39b5c0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:53.710488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5b6247-c7c2-489d-a429-ce466a39b5c0 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:53.711792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 568/1111] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:53.717388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:53.717801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:53.717938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:53.736825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:53.738000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:53.802063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 544/1112] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:54.070975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b6247-c7c2-489d-a429-ce466a39b5c0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:54.169868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5b6247-c7c2-489d-a429-ce466a39b5c0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:54.206149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:54.206803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:54.207166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:54.207445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] PATH_INFO: `/v3/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:54.209517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] RBAC: Authorizing `identity:get_role(role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:54.210935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:54.212263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 569/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:54 2022] GET /identity/v3/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:54.219023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:54.312920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a141301b-3b4b-49cf-9c6c-de853f3ceed7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:54.321748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:54.347909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:54.348520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:54.348793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:54.349064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] PATH_INFO: `/v3/users/fe94efc7307b4bcda0bf7f11513c64a5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:54.362197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] RBAC: Authorizing `identity:get_user(user_id=fe94efc7307b4bcda0bf7f11513c64a5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:54.363643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:54.365236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 545/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:54 2022] GET /identity/v3/users/fe94efc7307b4bcda0bf7f11513c64a5 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:54.371023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:54.424974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:54.468902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18896108-62d7-47d3-8f9b-ef7dcbf01db3 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:54.501219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:54.501807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:54.502080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:54.502377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:54.566003 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:54.567563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:54.619664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:54.663612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 570/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:08:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:08:54.669506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:54.764900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b0b93-e0b3-451d-9383-688989f2aa82 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:54.797514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10cf978-00e4-430c-acd0-8a0199b977a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:54.798120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf978-00e4-430c-acd0-8a0199b977a3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:54.798418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf978-00e4-430c-acd0-8a0199b977a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:54.798688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10cf978-00e4-430c-acd0-8a0199b977a3 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:54.799777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf978-00e4-430c-acd0-8a0199b977a3 None admin] RBAC: Authorizing `identity:get_project(project_id=54a1fb95d53148ba92c040d07ca973f7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:54.801173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10cf978-00e4-430c-acd0-8a0199b977a3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:54.802792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 546/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:08:54 2022] GET /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:54.808838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:54.901824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2260b-81b1-4348-b7a4-f815526e6d26 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:54.934212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:54.934819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:54.935094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:54.935362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:54.936571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] RBAC: Adding query filter params (role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=54a1fb95d53148ba92c040d07ca973f7, user.id=fe94efc7307b4bcda0bf7f11513c64a5) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:08:54.936873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:54.938315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4653e411-1ae8-4279-aefe-43bf18096d53 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:54.943506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 571/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:08:54 2022] GET /identity/v3/role_assignments?user.id=fe94efc7307b4bcda0bf7f11513c64a5&scope.project.id=54a1fb95d53148ba92c040d07ca973f7&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:57.045449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c8c5-92cb-4f2d-b188-f1ba870dc65f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:57.045956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c8c5-92cb-4f2d-b188-f1ba870dc65f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:57.045956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a1c8c5-92cb-4f2d-b188-f1ba870dc65f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:57.048028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 547/1118] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:08:57.053043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:57.053431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:57.053581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:57.071127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:08:57.072288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:08:57.084732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:57.136410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 572/1119] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:08:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:08:57.408714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1c8c5-92cb-4f2d-b188-f1ba870dc65f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:57.443235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:57.506724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a1c8c5-92cb-4f2d-b188-f1ba870dc65f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:57.541391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:57.542013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:57.542313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:57.542587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] PATH_INFO: `/v3/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:57.543946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] RBAC: Authorizing `identity:get_role(role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:57.545957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:57.547321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 548/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:57 2022] GET /identity/v3/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:57.553130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:57.649165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4577e2-6560-4c5d-a6db-c17a9bc3163f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:57.682810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:57.683359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:57.683637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:57.683908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] PATH_INFO: `/v3/users/fe94efc7307b4bcda0bf7f11513c64a5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:57.685009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] RBAC: Authorizing `identity:get_user(user_id=fe94efc7307b4bcda0bf7f11513c64a5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:57.687070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:57.688654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 573/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:08:57 2022] GET /identity/v3/users/fe94efc7307b4bcda0bf7f11513c64a5 => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:57.694450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:57.788960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bebb4a-75f8-4e26-ba77-43917fcb55bf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:57.790441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:57.822299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11e1eb1-8706-4754-837c-2da7ff32f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:57.822903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e1eb1-8706-4754-837c-2da7ff32f390 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:57.823179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e1eb1-8706-4754-837c-2da7ff32f390 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:57.823446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e1eb1-8706-4754-837c-2da7ff32f390 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:57.824514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e1eb1-8706-4754-837c-2da7ff32f390 None admin] RBAC: Authorizing `identity:get_project(project_id=54a1fb95d53148ba92c040d07ca973f7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:57.825936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11e1eb1-8706-4754-837c-2da7ff32f390 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:57.827560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 549/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:08:57 2022] GET /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:08:57.833376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:08:57.851680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:08:57.937096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a4d4f-213d-478c-b8f7-29d300dd425b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:08:57.970489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e90da83-1566-4ccc-9c47-46bd4b24eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:08:57.971127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90da83-1566-4ccc-9c47-46bd4b24eee1 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:08:57.971402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90da83-1566-4ccc-9c47-46bd4b24eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:08:57.971666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e90da83-1566-4ccc-9c47-46bd4b24eee1 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7/users/fe94efc7307b4bcda0bf7f11513c64a5/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:08:57.973760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90da83-1566-4ccc-9c47-46bd4b24eee1 None admin] RBAC: Authorizing `identity:create_grant(project_id=54a1fb95d53148ba92c040d07ca973f7, user_id=fe94efc7307b4bcda0bf7f11513c64a5, role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:08:57.975206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e90da83-1566-4ccc-9c47-46bd4b24eee1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:08:57.988011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 574/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:08:57 2022] PUT /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7/users/fe94efc7307b4bcda0bf7f11513c64a5/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:09:00.116050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8fef2-04a0-4cdb-9cf1-dacf139a2b67 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:00.116551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8fef2-04a0-4cdb-9cf1-dacf139a2b67 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:00.116659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8fef2-04a0-4cdb-9cf1-dacf139a2b67 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:00.117941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 550/1124] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:00.123972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:00.124370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:00.124532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:00.142769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:00.143942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:00.207889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 575/1125] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:00.478594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8fef2-04a0-4cdb-9cf1-dacf139a2b67 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:00.575490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8fef2-04a0-4cdb-9cf1-dacf139a2b67 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:00.606758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:00.608153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:00.608513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:00.609146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] PATH_INFO: `/v3/roles/0019fabba1fc4d229f9f785e32272bb1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:00.610537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] RBAC: Authorizing `identity:get_role(role_id=0019fabba1fc4d229f9f785e32272bb1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:00.611874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:00.613146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 551/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:00 2022] GET /identity/v3/roles/0019fabba1fc4d229f9f785e32272bb1 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:00.618969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:00.628398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:00.721206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d319d37-90d9-4e55-bfe8-29b7c31076ca None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:00.757484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:00.758041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:00.758352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:00.758625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] PATH_INFO: `/v3/users/fe94efc7307b4bcda0bf7f11513c64a5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:00.759739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] RBAC: Authorizing `identity:get_user(user_id=fe94efc7307b4bcda0bf7f11513c64a5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:00.761174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:00.762794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 576/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:00 2022] GET /identity/v3/users/fe94efc7307b4bcda0bf7f11513c64a5 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:00.768889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:00.807801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:00.865254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ba2e0-391a-4b32-bb10-3358f6f34b0a None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:00.896977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:00.897567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:00.897840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:00.898110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:00.961290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:00.963255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:00.999598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:01.056846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 552/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:09:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:09:01.062881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:01.136191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:01.161545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a139465-5d5a-4868-9eef-60299ae039ba None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:01.192529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954605ec-b54c-41e3-a1fc-c264a10978c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:01.193140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954605ec-b54c-41e3-a1fc-c264a10978c4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:01.193414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954605ec-b54c-41e3-a1fc-c264a10978c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:01.193680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954605ec-b54c-41e3-a1fc-c264a10978c4 None admin] PATH_INFO: `/v3/projects/54a1fb95d53148ba92c040d07ca973f7` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:01.194782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954605ec-b54c-41e3-a1fc-c264a10978c4 None admin] RBAC: Authorizing `identity:get_project(project_id=54a1fb95d53148ba92c040d07ca973f7)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:01.196196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954605ec-b54c-41e3-a1fc-c264a10978c4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:01.197791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 577/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:01 2022] GET /identity/v3/projects/54a1fb95d53148ba92c040d07ca973f7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:01.204442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:01.299229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae2e4f6-7c3c-4fc4-830d-a5b887ccecb3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:01.331064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:01.331592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:01.331866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:01.332135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:01.332839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] RBAC: Adding query filter params (role.id=0019fabba1fc4d229f9f785e32272bb1, scope.project.id=54a1fb95d53148ba92c040d07ca973f7, user.id=fe94efc7307b4bcda0bf7f11513c64a5) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:01.333143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:01.334574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd8226-3124-4abc-8f18-bb3de94843c6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:01.340307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 553/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:09:01 2022] GET /identity/v3/role_assignments?user.id=fe94efc7307b4bcda0bf7f11513c64a5&scope.project.id=54a1fb95d53148ba92c040d07ca973f7&role.id=0019fabba1fc4d229f9f785e32272bb1 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:03.480336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d271fc-3de0-45bb-b531-2f1466402e1d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:03.480864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d271fc-3de0-45bb-b531-2f1466402e1d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:03.480864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d271fc-3de0-45bb-b531-2f1466402e1d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:03.482752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 578/1131] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:03.488229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:03.488649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:03.488780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:03.495079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:03.508457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:03.509632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:03.573042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 554/1132] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:09:03.847577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d271fc-3de0-45bb-b531-2f1466402e1d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:03.949072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d271fc-3de0-45bb-b531-2f1466402e1d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:03.984546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2082385-1d81-411c-8194-3ba474a4677b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:03.985159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2082385-1d81-411c-8194-3ba474a4677b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:03.985439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2082385-1d81-411c-8194-3ba474a4677b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:03.985711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2082385-1d81-411c-8194-3ba474a4677b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:03.986813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2082385-1d81-411c-8194-3ba474a4677b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:03.988236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2082385-1d81-411c-8194-3ba474a4677b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:03.989779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 579/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:03.995703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:04.090673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52face-cab6-4c50-b4a8-f318d88e596a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:04.123621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:04.128854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e784c73e-2bed-48db-a92f-0185931364e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:04.129453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c73e-2bed-48db-a92f-0185931364e7 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:04.129728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c73e-2bed-48db-a92f-0185931364e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:04.129997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e784c73e-2bed-48db-a92f-0185931364e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:04.130666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c73e-2bed-48db-a92f-0185931364e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:04.132069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e784c73e-2bed-48db-a92f-0185931364e7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:04.157777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 555/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:09:03 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:09:06.316354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24fe13-4e36-4ddc-9067-0ef6977029fe None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:06.316916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24fe13-4e36-4ddc-9067-0ef6977029fe None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:06.316916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24fe13-4e36-4ddc-9067-0ef6977029fe None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:06.318229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 580/1135] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:06.323489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:06.323896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:06.324040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:06.342674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:06.343862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:06.407790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 556/1136] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:06.678383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24fe13-4e36-4ddc-9067-0ef6977029fe None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:06.690507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:06.776968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24fe13-4e36-4ddc-9067-0ef6977029fe None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:06.809333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268b7996-3480-4a2a-b458-8024340548a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:06.810452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b7996-3480-4a2a-b458-8024340548a2 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:06.810744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b7996-3480-4a2a-b458-8024340548a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:06.811010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268b7996-3480-4a2a-b458-8024340548a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:06.812094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b7996-3480-4a2a-b458-8024340548a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:06.813500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268b7996-3480-4a2a-b458-8024340548a2 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:06.815063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 581/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:09:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:06.820828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:06.914330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d89889-1db4-4c6a-ae33-5c6755e76b11 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:06.927328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:06.948108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67347ddb-7216-4cc9-a009-8596cab34a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:06.948694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347ddb-7216-4cc9-a009-8596cab34a39 None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:06.948967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347ddb-7216-4cc9-a009-8596cab34a39 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:06.949232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67347ddb-7216-4cc9-a009-8596cab34a39 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:06.949868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67347ddb-7216-4cc9-a009-8596cab34a39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:06.951439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67347ddb-7216-4cc9-a009-8596cab34a39 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:06.972429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 557/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:09:06 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:09:09.126645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54cae4-3711-4502-8f21-151b33728334 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:09.127155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54cae4-3711-4502-8f21-151b33728334 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:09.127247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c54cae4-3711-4502-8f21-151b33728334 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:09.128520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 582/1139] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:09.133589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:09.134071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:09.134174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:09.152808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:09.154233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:09.219175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 558/1140] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:09.491027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54cae4-3711-4502-8f21-151b33728334 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:09.567528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:09.590331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c54cae4-3711-4502-8f21-151b33728334 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:09.622575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:09.623179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:09.623452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:09.623724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:09.633572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:09:09.639774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:09.640897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:09:09.642025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 583/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:09.646890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:09.742454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:09.745246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4243fc-8e7c-46c0-a15b-18026404b2db None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:09.778998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:09.779587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:09.779873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:09.780146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:09.780800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:09.781103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:09.782482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:09.790149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 559/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:09.796125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:09.897668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55260a05-f959-4350-86b3-f7ff8eccccd9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:09.930025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:09.930672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:09.930952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:09.931225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] PATH_INFO: `/v3/users/16700220fbcc41ab962f2614a45c4b8b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:09.939310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:09.945194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=16700220fbcc41ab962f2614a45c4b8b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:09.946674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:09.948271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 584/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:09 2022] GET /identity/v3/users/16700220fbcc41ab962f2614a45c4b8b => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:09.954834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:10.048452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d281cde-4bd4-4bc1-ac08-47d3f250ee83 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:10.080606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:10.081182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:10.081455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:10.081721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:10.110506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:10.146805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:10.148221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:10.241242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 560/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:09:09 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:09:10.247173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:10.349687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4137912-4e6b-4c6f-83c0-4ccb33665ac6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:10.382029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d82dea0-9411-432b-9d1a-c0b77e106d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:10.382667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82dea0-9411-432b-9d1a-c0b77e106d28 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:10.382939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82dea0-9411-432b-9d1a-c0b77e106d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:10.383201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d82dea0-9411-432b-9d1a-c0b77e106d28 None admin] PATH_INFO: `/v3/projects/64bf00349a12432aabdf5f2a54db9a7f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:10.384275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82dea0-9411-432b-9d1a-c0b77e106d28 None admin] RBAC: Authorizing `identity:get_project(project_id=64bf00349a12432aabdf5f2a54db9a7f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:10.385682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d82dea0-9411-432b-9d1a-c0b77e106d28 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:10.387296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 585/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:10 2022] GET /identity/v3/projects/64bf00349a12432aabdf5f2a54db9a7f => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:10.394079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:10.452724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:10.489998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d3e1f-22c7-4053-8a02-23af02d65960 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:10.522582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:10.523169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:10.523445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:10.523715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:10.524422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=64bf00349a12432aabdf5f2a54db9a7f, user.id=16700220fbcc41ab962f2614a45c4b8b) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:10.524729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:10.526123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea06bb1-90aa-4d62-9a6b-bfb652389f64 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:10.531317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 561/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:09:10 2022] GET /identity/v3/role_assignments?user.id=16700220fbcc41ab962f2614a45c4b8b&scope.project.id=64bf00349a12432aabdf5f2a54db9a7f&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:12.679681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3fc0d-3a6f-4fa4-afab-ade2c6b35b5d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:12.680230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3fc0d-3a6f-4fa4-afab-ade2c6b35b5d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:12.680654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b3fc0d-3a6f-4fa4-afab-ade2c6b35b5d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:12.682511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 586/1147] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:12.688766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:12.689214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:12.689307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:12.707472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:12.708848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:12.775756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 562/1148] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:13.053838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3fc0d-3a6f-4fa4-afab-ade2c6b35b5d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:13.114861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:13.153043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b3fc0d-3a6f-4fa4-afab-ade2c6b35b5d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:13.187570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:13.188204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:13.188483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:13.188754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:13.200121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:09:13.205062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:13.206164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:09:13.207058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 587/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:09:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:13.212335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:13.265692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:13.307324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77006417-3b62-40dc-b40c-57126aa5541c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:13.339959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:13.340551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:13.340852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:13.341193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:13.341845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:13.342140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:13.343503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:13.351171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 563/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:09:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:13.357531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:13.451746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d19069-4e81-4c7e-8f75-1dbf9405f578 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:13.475164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:13.488609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:13.489417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:13.489690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:13.489958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] PATH_INFO: `/v3/users/16700220fbcc41ab962f2614a45c4b8b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:13.491093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] RBAC: Authorizing `identity:get_user(user_id=16700220fbcc41ab962f2614a45c4b8b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:13.492726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:13.494332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 588/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:13 2022] GET /identity/v3/users/16700220fbcc41ab962f2614a45c4b8b => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:13.500436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:13.596389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5b5e27-ed78-463c-8fb9-0cbb4e816137 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:13.612546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:13.629022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecf6631-ec60-471e-903e-a94f24fc8f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:13.629643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6631-ec60-471e-903e-a94f24fc8f6c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:13.629919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6631-ec60-471e-903e-a94f24fc8f6c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:13.630197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecf6631-ec60-471e-903e-a94f24fc8f6c None admin] PATH_INFO: `/v3/projects/64bf00349a12432aabdf5f2a54db9a7f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:13.631327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6631-ec60-471e-903e-a94f24fc8f6c None admin] RBAC: Authorizing `identity:get_project(project_id=64bf00349a12432aabdf5f2a54db9a7f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:13.632737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fecf6631-ec60-471e-903e-a94f24fc8f6c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:13.634383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 564/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:13 2022] GET /identity/v3/projects/64bf00349a12432aabdf5f2a54db9a7f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:13.640432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:13.737251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deaa389b-6ccf-4abe-83b7-8310ea4f7803 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:13.768914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5901a1c6-c0aa-4855-9ce9-a6cac8fe7727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:13.769551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901a1c6-c0aa-4855-9ce9-a6cac8fe7727 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:13.769827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901a1c6-c0aa-4855-9ce9-a6cac8fe7727 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:13.770105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5901a1c6-c0aa-4855-9ce9-a6cac8fe7727 None admin] PATH_INFO: `/v3/projects/64bf00349a12432aabdf5f2a54db9a7f/users/16700220fbcc41ab962f2614a45c4b8b/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:13.771989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901a1c6-c0aa-4855-9ce9-a6cac8fe7727 None admin] RBAC: Authorizing `identity:create_grant(project_id=64bf00349a12432aabdf5f2a54db9a7f, user_id=16700220fbcc41ab962f2614a45c4b8b, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:13.773410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5901a1c6-c0aa-4855-9ce9-a6cac8fe7727 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:13.789421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 589/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:09:13 2022] PUT /identity/v3/projects/64bf00349a12432aabdf5f2a54db9a7f/users/16700220fbcc41ab962f2614a45c4b8b/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:09:15.886528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6897cda-fd77-4540-a8f3-bbea3415d83e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:15.887019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6897cda-fd77-4540-a8f3-bbea3415d83e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:15.887019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6897cda-fd77-4540-a8f3-bbea3415d83e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:15.888486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 565/1154] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:15.893555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:15.893993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:15.894086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:15.912401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:15.913564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:15.958620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:15.978921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 590/1155] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:16.245757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6897cda-fd77-4540-a8f3-bbea3415d83e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:16.343595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6897cda-fd77-4540-a8f3-bbea3415d83e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:16.376284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:16.376895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:16.377170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:16.377440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:16.386933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:09:16.392125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:16.393215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:09:16.394091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 566/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:09:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:16.399146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:16.497907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6f82-fdcc-4523-ac63-7ca4541cb4d5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:16.530191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:16.530811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:16.531089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:16.531359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:16.532014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:16.532316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:16.533670 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:16.541284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 591/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:09:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:16.547820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:16.593089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:16.645021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbc64e3-d567-4e64-8885-1609609ca29e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:16.677731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:16.678369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:16.678645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:16.678915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] PATH_INFO: `/v3/users/16700220fbcc41ab962f2614a45c4b8b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:16.680003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] RBAC: Authorizing `identity:get_user(user_id=16700220fbcc41ab962f2614a45c4b8b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:16.681459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:16.683063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 567/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:16 2022] GET /identity/v3/users/16700220fbcc41ab962f2614a45c4b8b => generated 286 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:16.689025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:16.734969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:16.787126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9d3a34-775f-4e12-8bd9-028cbac3a04b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:16.819229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:16.820102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:16.820382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:16.820649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:16.885045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:16.886501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:16.928757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:16.982127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 592/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:09:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 04 10:09:16.988111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:17.083645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c113e-67bd-4dee-b344-c0ed24292c19 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:17.097210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:17.116450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706a373-8dac-4140-8031-a53422445886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:17.117068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706a373-8dac-4140-8031-a53422445886 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:17.117346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706a373-8dac-4140-8031-a53422445886 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:17.117619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706a373-8dac-4140-8031-a53422445886 None admin] PATH_INFO: `/v3/projects/64bf00349a12432aabdf5f2a54db9a7f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:17.118749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706a373-8dac-4140-8031-a53422445886 None admin] RBAC: Authorizing `identity:get_project(project_id=64bf00349a12432aabdf5f2a54db9a7f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:17.120155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706a373-8dac-4140-8031-a53422445886 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:17.121751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 568/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:16 2022] GET /identity/v3/projects/64bf00349a12432aabdf5f2a54db9a7f => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:17.127921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:17.239385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5746e781-cbcd-453d-b97e-592e65c5d5bf None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:17.273197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:17.273786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:17.274062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:17.274363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:17.275083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=64bf00349a12432aabdf5f2a54db9a7f, user.id=16700220fbcc41ab962f2614a45c4b8b) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:17.275388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:17.276798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7879bed9-d80d-42b2-aacf-cb89e0ba6da1 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:17.282487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 593/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:09:17 2022] GET /identity/v3/role_assignments?user.id=16700220fbcc41ab962f2614a45c4b8b&scope.project.id=64bf00349a12432aabdf5f2a54db9a7f&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:19.424421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c66d-5009-41c2-be6b-fbbae7ebd928 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:19.425020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c66d-5009-41c2-be6b-fbbae7ebd928 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:19.425020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3636c66d-5009-41c2-be6b-fbbae7ebd928 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:19.426353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 569/1162] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:19.431668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:19.432095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:19.432215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:19.450476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:19.451642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:19.453533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:19.516426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 594/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:19.782912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636c66d-5009-41c2-be6b-fbbae7ebd928 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:19.879620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3636c66d-5009-41c2-be6b-fbbae7ebd928 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:19.911706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:19.912292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:19.912567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:19.912838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:19.958946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:19.976543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:19.977960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:20.071701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 570/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:09:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:09:20.078072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:20.172727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d1a771-d4f5-4859-9534-ef40ef3b4439 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:20.205213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:20.205825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:20.206101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:20.206399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:20.213091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:20.214553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:20.216715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-55ed34ea-85d9-4ac2-a220-a57c7e4c27cf None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:20.227208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:20.233739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:09:20.234705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 595/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:09:20 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:20.235603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:20.311511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:20.328863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e798e49c-82f7-4ebf-bf58-8b34ab8f94db None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:20.362858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:20.363450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:20.363728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:20.363999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:20.364622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:20.364920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:20.366289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc27b72-c068-46f0-8ed1-37fbc2e27c05 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:20.373806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 571/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Tue Oct 4 10:09:20 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:22.484484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abded4-0cc0-40a0-9546-6c2276cf398b None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:22.485029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abded4-0cc0-40a0-9546-6c2276cf398b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:22.485029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2abded4-0cc0-40a0-9546-6c2276cf398b None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:22.486407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 596/1167] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:22.491620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79270f-461e-4491-8023-9db77cb751d0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:22.492020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79270f-461e-4491-8023-9db77cb751d0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:22.492172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e79270f-461e-4491-8023-9db77cb751d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:22.509837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7e79270f-461e-4491-8023-9db77cb751d0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:22.511032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e79270f-461e-4491-8023-9db77cb751d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:22.573487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 572/1168] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:22.847235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abded4-0cc0-40a0-9546-6c2276cf398b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:22.945832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2abded4-0cc0-40a0-9546-6c2276cf398b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:22.978880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f7e9bd-010b-41d3-a1b4-5c148f8b1282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:22.979470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7e9bd-010b-41d3-a1b4-5c148f8b1282 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:22.979745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7e9bd-010b-41d3-a1b4-5c148f8b1282 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:22.980012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f7e9bd-010b-41d3-a1b4-5c148f8b1282 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:22.980670 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7e9bd-010b-41d3-a1b4-5c148f8b1282 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:22.982786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f7e9bd-010b-41d3-a1b4-5c148f8b1282 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:22.999425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 597/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Tue Oct 4 10:09:22 2022] POST /identity/v3/domains => generated 267 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:09:25.150508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b50695-ac8b-4a3f-b3fe-1e2bb15a17a4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:25.151045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b50695-ac8b-4a3f-b3fe-1e2bb15a17a4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:25.151045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b50695-ac8b-4a3f-b3fe-1e2bb15a17a4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:25.152380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 573/1170] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:25.157863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:25.158279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:25.158569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:25.176613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:25.177775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:25.229617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:25.244329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 598/1171] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:25.509640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b50695-ac8b-4a3f-b3fe-1e2bb15a17a4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:25.582840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:25.606097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b50695-ac8b-4a3f-b3fe-1e2bb15a17a4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:25.639933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f010bb5f-2263-434b-b664-eb8b875560bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:25.640555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010bb5f-2263-434b-b664-eb8b875560bf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:25.640831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010bb5f-2263-434b-b664-eb8b875560bf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:25.641104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f010bb5f-2263-434b-b664-eb8b875560bf None admin] PATH_INFO: `/v3/domains/451306d1570d411d994c50d51074df10` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:25.651087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010bb5f-2263-434b-b664-eb8b875560bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=451306d1570d411d994c50d51074df10)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:25.652499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f010bb5f-2263-434b-b664-eb8b875560bf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:25.654255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 574/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Tue Oct 4 10:09:25 2022] GET /identity/v3/domains/451306d1570d411d994c50d51074df10 => generated 267 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:25.659990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:25.754824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f1bb2d-f1b9-49a7-b6b7-9b5da6c91f8f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:25.787080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a6c74-0b5e-4f80-9a50-6aca169d8213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:25.787607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6c74-0b5e-4f80-9a50-6aca169d8213 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:25.787880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6c74-0b5e-4f80-9a50-6aca169d8213 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:25.788147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a6c74-0b5e-4f80-9a50-6aca169d8213 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:25.788775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a6c74-0b5e-4f80-9a50-6aca169d8213 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:25.790187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a6c74-0b5e-4f80-9a50-6aca169d8213 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:25.816813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 599/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:09:25 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:09:27.972170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1a98-6f30-4bb8-b254-fa104cd70df4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:27.972681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1a98-6f30-4bb8-b254-fa104cd70df4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:27.972681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c1a98-6f30-4bb8-b254-fa104cd70df4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:27.973957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 575/1174] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:27.979208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:27.979588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:27.979748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:27.997915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:27.999115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:28.031783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:28.064301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 600/1175] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:28.331591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1a98-6f30-4bb8-b254-fa104cd70df4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:28.433598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c1a98-6f30-4bb8-b254-fa104cd70df4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:28.468752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed8b8fe-bacc-42d3-96ed-c3552a331348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:28.469367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b8fe-bacc-42d3-96ed-c3552a331348 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:28.469638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b8fe-bacc-42d3-96ed-c3552a331348 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:28.469905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed8b8fe-bacc-42d3-96ed-c3552a331348 None admin] PATH_INFO: `/v3/domains/451306d1570d411d994c50d51074df10` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:28.471012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b8fe-bacc-42d3-96ed-c3552a331348 None admin] RBAC: Authorizing `identity:get_domain(domain_id=451306d1570d411d994c50d51074df10)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:28.472417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed8b8fe-bacc-42d3-96ed-c3552a331348 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:28.473955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 576/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Tue Oct 4 10:09:28 2022] GET /identity/v3/domains/451306d1570d411d994c50d51074df10 => generated 267 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:28.479702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:28.574125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f5f865-e5d4-445c-a1e9-d1324920118d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:28.607360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a8a9bc-a05c-45b0-a8da-896690f2eeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:28.607958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a9bc-a05c-45b0-a8da-896690f2eeb0 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:28.608234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a9bc-a05c-45b0-a8da-896690f2eeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:28.608502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a8a9bc-a05c-45b0-a8da-896690f2eeb0 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:28.609150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8a9bc-a05c-45b0-a8da-896690f2eeb0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:28.610590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a8a9bc-a05c-45b0-a8da-896690f2eeb0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:28.632394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 601/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:09:28 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:09:30.803402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e517a-d17e-4bd5-9b98-27f400bd9c7b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:30.803910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e517a-d17e-4bd5-9b98-27f400bd9c7b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:30.803910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19e517a-d17e-4bd5-9b98-27f400bd9c7b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:30.805267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 577/1178] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:30.811242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:30.811674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:30.811774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:30.830704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:30.831879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:30.851304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:30.898407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 602/1179] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:09:31.175744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e517a-d17e-4bd5-9b98-27f400bd9c7b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:31.179524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:31.274824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19e517a-d17e-4bd5-9b98-27f400bd9c7b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:31.306785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:31.307397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:31.307675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:31.307949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:31.318073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:09:31.322758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:31.323845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:09:31.324844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 578/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:31.329879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:31.423499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ea248-6145-46dc-b5e7-d6d2628730d2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:31.457059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:31.457589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:31.457862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:31.458130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:31.458806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:31.459103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:31.460452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:31.469277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 603/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:31.475105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:31.535518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:31.571922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b31da-3ff0-4249-9600-ab22392da673 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:31.603931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:31.604533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:31.604806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:31.605074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] PATH_INFO: `/v3/users/9aa0a6f0622843d9b5c3845670f92052` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:31.618235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] RBAC: Authorizing `identity:get_user(user_id=9aa0a6f0622843d9b5c3845670f92052)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:31.619677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:31.621297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 579/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:31 2022] GET /identity/v3/users/9aa0a6f0622843d9b5c3845670f92052 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:31.627099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:31.646038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:31.723973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaaa6d2-4de5-4426-81cf-7ab34c4e23e0 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:31.756554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:31.757161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:31.757434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:31.757737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:31.824597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:31.826042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:31.843467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:31.920814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 604/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:09:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:09:31.926787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:32.021129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc5412a-3ead-4d73-a495-51bb994aaf4f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:32.044032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:32.055529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf5363b-57a0-4b9f-bb16-47f843d2a7fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:32.056149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5363b-57a0-4b9f-bb16-47f843d2a7fb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:32.056420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5363b-57a0-4b9f-bb16-47f843d2a7fb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:32.056698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf5363b-57a0-4b9f-bb16-47f843d2a7fb None admin] PATH_INFO: `/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:32.057778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5363b-57a0-4b9f-bb16-47f843d2a7fb None admin] RBAC: Authorizing `identity:get_project(project_id=22689d4d323c4584ab1971f3c1d7f2dd)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:32.059888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf5363b-57a0-4b9f-bb16-47f843d2a7fb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:32.061491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 580/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:31 2022] GET /identity/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:32.067597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:32.162622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1eb0ac-9cae-4fad-822f-be3a9c4847f6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:32.180044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:32.197224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:32.197819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:32.198093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:32.198397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:32.199113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=22689d4d323c4584ab1971f3c1d7f2dd, user.id=9aa0a6f0622843d9b5c3845670f92052) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:32.199579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:32.201018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c785198e-10aa-4bcd-88dd-8fbb7864879f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:32.206217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 605/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:09:32 2022] GET /identity/v3/role_assignments?user.id=9aa0a6f0622843d9b5c3845670f92052&scope.project.id=22689d4d323c4584ab1971f3c1d7f2dd&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:34.333520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b249-d8f3-4c85-8bfe-0f55a382befa None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:34.334048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b249-d8f3-4c85-8bfe-0f55a382befa None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:34.334048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5257b249-d8f3-4c85-8bfe-0f55a382befa None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:34.335462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 581/1186] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:34.341022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:34.341436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:34.341587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:34.359666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:34.360845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:34.430145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 606/1187] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:34.703234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257b249-d8f3-4c85-8bfe-0f55a382befa None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:34.804952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5257b249-d8f3-4c85-8bfe-0f55a382befa None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:34.837829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:34.838470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:34.838749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:34.839020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:34.848648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:09:34.853347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:34.854489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:09:34.855358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 582/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:34.861121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:34.955201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431c908-7837-42d5-a04d-2639dffb5a27 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:34.967654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:34.988748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:34.989342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:34.989618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:34.989880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:34.990555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:34.990849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:34.992198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:35.000520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 607/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:35.006490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:35.054465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:35.101912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a41acc-10b3-4511-8ed6-70a6a2c3a9d3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:35.134295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:35.134903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:35.135183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:35.135456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] PATH_INFO: `/v3/users/9aa0a6f0622843d9b5c3845670f92052` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:35.136544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] RBAC: Authorizing `identity:get_user(user_id=9aa0a6f0622843d9b5c3845670f92052)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:35.138654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:35.140203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 583/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:35 2022] GET /identity/v3/users/9aa0a6f0622843d9b5c3845670f92052 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:35.145922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:35.246164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c7726e-b824-40ab-91af-50399ec5f6aa None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:35.279673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83395e50-7696-4443-9352-03cdd89c1c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:35.280293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395e50-7696-4443-9352-03cdd89c1c78 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:35.280570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395e50-7696-4443-9352-03cdd89c1c78 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:35.280840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83395e50-7696-4443-9352-03cdd89c1c78 None admin] PATH_INFO: `/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:35.281945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395e50-7696-4443-9352-03cdd89c1c78 None admin] RBAC: Authorizing `identity:get_project(project_id=22689d4d323c4584ab1971f3c1d7f2dd)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:35.283407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83395e50-7696-4443-9352-03cdd89c1c78 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:35.285023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 608/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:35 2022] GET /identity/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:35.290943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:35.387367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f27397-4df2-48a0-8869-faf279daee4d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:35.403769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:35.420739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe709d8-8635-42e9-bb98-8ff366a8e362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:35.421380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe709d8-8635-42e9-bb98-8ff366a8e362 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:35.421651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe709d8-8635-42e9-bb98-8ff366a8e362 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:35.421918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe709d8-8635-42e9-bb98-8ff366a8e362 None admin] PATH_INFO: `/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd/users/9aa0a6f0622843d9b5c3845670f92052/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:35.423786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe709d8-8635-42e9-bb98-8ff366a8e362 None admin] RBAC: Authorizing `identity:create_grant(project_id=22689d4d323c4584ab1971f3c1d7f2dd, user_id=9aa0a6f0622843d9b5c3845670f92052, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:35.425195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe709d8-8635-42e9-bb98-8ff366a8e362 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:35.437160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 584/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:09:35 2022] PUT /identity/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd/users/9aa0a6f0622843d9b5c3845670f92052/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:09:37.560536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12817a35-85ee-44a6-b256-f0334de7bce0 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:37.561068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12817a35-85ee-44a6-b256-f0334de7bce0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:37.561068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12817a35-85ee-44a6-b256-f0334de7bce0 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:37.562456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 609/1193] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:09:37.567856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:37.568281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:37.568400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:37.586038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:09:37.587491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:09:37.650919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 585/1194] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:09:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:09:37.923456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12817a35-85ee-44a6-b256-f0334de7bce0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:37.967192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:38.021747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12817a35-85ee-44a6-b256-f0334de7bce0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:38.054107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:38.054746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:38.055021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:38.055290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:38.065145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:09:38.069874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:09:38.070970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:09:38.072129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 610/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:09:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:09:38.077445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:38.172139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4199fc1-b2b5-4eb1-9787-db0307f83a7c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:38.181414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:38.207369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:38.207969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:38.208246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:38.208516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:38.209163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:38.209462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:38.210840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:38.219486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 586/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:38.225421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:38.320657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd4e6c-fca4-4973-8f52-699b97daef3b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:38.322124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:38.354721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:38.355355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:38.355623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:38.355891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] PATH_INFO: `/v3/users/9aa0a6f0622843d9b5c3845670f92052` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:38.356975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] RBAC: Authorizing `identity:get_user(user_id=9aa0a6f0622843d9b5c3845670f92052)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:38.358406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:38.359954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 611/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Tue Oct 4 10:09:38 2022] GET /identity/v3/users/9aa0a6f0622843d9b5c3845670f92052 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:38.365721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:38.469313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fc32b4-dcf5-4f3e-905e-88a15ef84732 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:38.501702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:38.502329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:38.502611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:38.502887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:38.540735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:38.568777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:38.570236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:38.666102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 587/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:09:38 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:09:38.672285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:38.768558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6570be5e-d641-4720-9304-60456c1f15ae None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:38.800883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012a277-47ac-4d9b-bb30-74221abe3929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:38.801493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a277-47ac-4d9b-bb30-74221abe3929 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:38.801769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a277-47ac-4d9b-bb30-74221abe3929 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:38.802038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012a277-47ac-4d9b-bb30-74221abe3929 None admin] PATH_INFO: `/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:38.803161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012a277-47ac-4d9b-bb30-74221abe3929 None admin] RBAC: Authorizing `identity:get_project(project_id=22689d4d323c4584ab1971f3c1d7f2dd)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:38.804583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c012a277-47ac-4d9b-bb30-74221abe3929 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:38.806187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 612/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Tue Oct 4 10:09:38 2022] GET /identity/v3/projects/22689d4d323c4584ab1971f3c1d7f2dd => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:09:38.812375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:09:38.878249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:38.908954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828fe86-a433-40bf-b906-0717895ca935 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:09:38.940615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:09:38.941198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:09:38.941468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:09:38.941734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:09:38.942466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=22689d4d323c4584ab1971f3c1d7f2dd, user.id=9aa0a6f0622843d9b5c3845670f92052) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:09:38.942770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:09:38.944173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8913b16b-2c55-4031-8e21-27e56d2185a6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:09:38.950380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 588/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:09:38 2022] GET /identity/v3/role_assignments?user.id=9aa0a6f0622843d9b5c3845670f92052&scope.project.id=22689d4d323c4584ab1971f3c1d7f2dd&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:09:48.796868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=81 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:09:48.947232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=33 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:34.752308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804861d-6d50-43d6-b28b-8323c7286094 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:34.752891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804861d-6d50-43d6-b28b-8323c7286094 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:34.752891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804861d-6d50-43d6-b28b-8323c7286094 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:34.754559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 613/1201] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:10:34.761089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:34.761591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:34.761695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:34.788638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:10:34.790027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:10:34.890474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 589/1202] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:10:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:10:35.180715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804861d-6d50-43d6-b28b-8323c7286094 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:35.311264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804861d-6d50-43d6-b28b-8323c7286094 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:35.356930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:35.357553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:35.357831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:35.358146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:35.370386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:35.371941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:35.388691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:35.389811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:35.390841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:35.390841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:35.390841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:35.395335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 614/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:35.396482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:35.514172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e7284-ec17-41b6-91e0-4ddc45adaf8b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:35.551455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:35.559045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:35.560119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:35.560791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:35.560791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:35.561237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:35.561543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:35.563064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5367022-2e70-4828-a653-e75b54c4003b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:35.586272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 590/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:35.589035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:35.665730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:35.732816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d79dc2-bbcd-4a1a-be32-8372cacc3514 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:35.779356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9986f06b-2965-4504-8fc8-48cf9638e8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:35.779976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986f06b-2965-4504-8fc8-48cf9638e8f8 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:35.780244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986f06b-2965-4504-8fc8-48cf9638e8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:35.780426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9986f06b-2965-4504-8fc8-48cf9638e8f8 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:35.781074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986f06b-2965-4504-8fc8-48cf9638e8f8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:35.782966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9986f06b-2965-4504-8fc8-48cf9638e8f8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:35.818434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 615/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Tue Oct 4 10:10:35 2022] POST /identity/v3/users => generated 251 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:10:38.203310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce306-03cb-4f07-ac09-8a601b55e77f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:38.203912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce306-03cb-4f07-ac09-8a601b55e77f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:38.204001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ce306-03cb-4f07-ac09-8a601b55e77f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:38.205643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 591/1206] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:10:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:10:38.213565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:38.214163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:38.214163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:38.236133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:10:38.237516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:10:38.319388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 616/1207] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:10:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:10:38.609897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ce306-03cb-4f07-ac09-8a601b55e77f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:38.730439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ce306-03cb-4f07-ac09-8a601b55e77f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:38.773869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:38.774617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:38.774919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:38.775188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:38.786786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:10:38.794439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:38.796345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:38.797328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 592/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:10:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:38.802700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:38.854066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:38.935668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dac0be1-1b86-415b-9a61-069fe365bf77 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:38.978136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:38.978793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:38.979084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:38.979333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:38.980077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:38.980398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:38.981915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:38.994241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 617/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:10:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:38.999543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:39.066577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:39.125776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8707e11-fc78-428a-a7d5-166ac209282b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:39.165883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:39.166625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:39.166896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:39.167190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:39.180956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:39.182596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:39.204954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:39.206449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:39.209720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:39.209720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:39.209720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:39.209720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 593/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:39.213409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:39.334052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:39.338584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd0a24d-860e-4097-bed8-1ba6d8a0c3f4 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:39.380252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:39.381084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:39.381366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:39.381633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:39.382590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:39.383009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:39.385575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:39.400807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 618/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:39.407391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:39.529136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b8f580-8826-45a9-8143-9bf3c62aac4b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:39.531189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:39.573493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:39.574158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:39.574459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:39.574728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] PATH_INFO: `/v3/users/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:39.582731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:39.584197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:10:39.596841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:39.597861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:39.599000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 594/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:39.607040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:39.738784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4b7998-5836-4e37-869d-74675b7a989b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:39.777302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:39.777935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:39.778228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:39.778505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:39.779368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:39.779595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:39.781121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:39.800717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 619/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Oct 4 10:10:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:10:39.807982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:39.934646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf53626-2c8e-4753-92a0-903f239832ea None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:39.976461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:39.977099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:39.977406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:39.977652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:40.016819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:40.059508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:40.061578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:40.182114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 595/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:10:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:10:40.189058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:40.210365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:40.299506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e958e-112e-4a89-9d9b-51435d145523 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:40.343662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:40.343662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:40.343662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:40.343662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:40.355241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:40.356911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:40.377820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:40.379608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:40.381086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:40.381086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:40.381086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:40.383171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 620/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:40.389905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:40.470760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:40.520625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c222b3-5b55-48b4-8003-a7c9d3c8a7cc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:40.563778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:40.564406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:40.564682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:40.564950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:40.565597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:40.565900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:40.567459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:40.581873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 596/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:40.590111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:40.678427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:40.722613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f410864-636c-4359-93b4-3fec72544078 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:40.764978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:40.765605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:40.765880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:40.766148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:40.775284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:40.777339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:10:40.792256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:40.793384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:40.794475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 621/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:40.799480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:40.907204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab6343-bb7b-4120-b779-b47c6b526660 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:40.931380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:40.944265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:40.944875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:40.945179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:40.945452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:40.946716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:40.947136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:40.948575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f854312-c77c-483a-9414-fd651a4dbbea None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:40.962054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 597/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:10:40.968785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:41.077726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df51999-bff8-4f70-a283-a12e031323bc None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:41.111233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:41.118646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:41.119254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:41.119533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:41.119807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:41.120536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=c873c7e9014b46c8b1bf717f1e0fe514) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:41.120890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:41.122381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4354a6b6-47f0-4ac0-99e3-4b31cf27643c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:41.127991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 622/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:10:40 2022] GET /identity/v3/role_assignments?user.id=c873c7e9014b46c8b1bf717f1e0fe514&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:43.354017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4983adb-5eba-4ae4-a4ff-dcd2bb6214ac None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:43.354595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4983adb-5eba-4ae4-a4ff-dcd2bb6214ac None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:43.354775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4983adb-5eba-4ae4-a4ff-dcd2bb6214ac None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:43.356249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 598/1220] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:10:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:10:43.361801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:43.362242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:43.362373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:43.380982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:10:43.382155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:10:43.464752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 623/1221] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:10:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 04 10:10:43.740721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4983adb-5eba-4ae4-a4ff-dcd2bb6214ac None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:43.856202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4983adb-5eba-4ae4-a4ff-dcd2bb6214ac None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:43.896516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:43.897096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:43.897442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:43.897778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:43.910093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:10:43.915438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:43.916615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:43.917640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 599/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:10:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:10:43.923560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:44.036176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755919b9-3be6-4f25-9c41-53d9b995c729 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:44.059789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:44.071778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:44.072617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:44.073021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:44.073322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:44.074136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] RBAC: Adding query filter params (name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:44.074466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:44.076175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:44.090806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 624/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:10:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:44.092198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:44.145311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:44.210237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aae803-19a3-4a25-a10c-6fcdde304e1c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:44.250940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:44.251594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:44.251937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:44.252207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:44.263026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:44.264464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:44.279139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:44.280374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:44.281261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:44.281261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:44.281261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:44.281261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 600/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:44.287376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:44.403340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71268d62-d881-421a-a22b-14e2ffda5b64 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:44.443300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:44.444135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:44.444485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:44.444779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:44.445504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:44.446389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:44.447744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:44.462105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 625/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:44.468263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:44.569330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:44.577037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0547c8fc-442f-4151-a87a-9722fb2402b7 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:44.619680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:44.620321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:44.620622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:44.620919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:44.629511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:44.630977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:10:44.642857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:44.644102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:44.645050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 601/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:44.651101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:44.699143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:44.757465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd8429-bc8a-4021-8738-36caad286aae None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:44.795505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:44.796205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:44.796515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:44.796846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:44.797713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:44.798082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:44.799716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:44.820513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 626/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Oct 4 10:10:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:44.827274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:44.937732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91261759-52a4-4b84-9617-9b16aba9675f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:44.975373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:44.976162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:44.976447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:44.976726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:44.988883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:44.990421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:45.005353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:45.006709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:45.007684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:45.007684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:45.007684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:45.007684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 602/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:10:45.013395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:45.140211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9945a4b-4377-4856-b828-02e04174f855 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:45.143078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:45.178306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:45.178914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:45.179191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:45.179464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:45.180912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:45.181223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:45.182634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:45.198248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 627/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:45.201749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:45.212814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:45.317632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e085bef-5401-48d3-bedc-5d9e54341dbc None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:45.354474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:45.355108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:45.355387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:45.355656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:45.364110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:45.365544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:10:45.377826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:45.378871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:45.379922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 603/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:45.386629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:45.509865 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b16ae45-e714-4ea5-8a29-fbe17c6102ed None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:45.554190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:45.555044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:45.555435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:45.555844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:45.556921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:45.557447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:45.560114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0dea44-a4b6-4f82-a80f-cea51b62ed96 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:45.573708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 628/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:45.580063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:45.660635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:45.699494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578cfe3e-b343-4383-b5f7-26fac540f0bf None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:45.747814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98066b3f-3477-41e5-813b-10b734430e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:45.749347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98066b3f-3477-41e5-813b-10b734430e71 None admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:45.749347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98066b3f-3477-41e5-813b-10b734430e71 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:45.749596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98066b3f-3477-41e5-813b-10b734430e71 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/c873c7e9014b46c8b1bf717f1e0fe514/roles/8b01b60bdf664e84ad21cc7e7d3ff607` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:45.767111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98066b3f-3477-41e5-813b-10b734430e71 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=c873c7e9014b46c8b1bf717f1e0fe514, role_id=8b01b60bdf664e84ad21cc7e7d3ff607)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:45.768555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98066b3f-3477-41e5-813b-10b734430e71 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:45.781810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 604/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:10:45 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/c873c7e9014b46c8b1bf717f1e0fe514/roles/8b01b60bdf664e84ad21cc7e7d3ff607 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:10:48.167905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f380e-c156-4da8-bb77-c962b0113c33 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:48.168449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f380e-c156-4da8-bb77-c962b0113c33 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:48.168449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f380e-c156-4da8-bb77-c962b0113c33 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:48.169792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 629/1233] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:10:48.175436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:48.175881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:48.176027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:48.197148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:10:48.198562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:10:48.280525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 605/1234] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:10:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:10:48.576586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f380e-c156-4da8-bb77-c962b0113c33 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:48.604134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:48.697631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f380e-c156-4da8-bb77-c962b0113c33 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:48.738390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:48.739021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:48.739296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:48.739567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:48.752219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:10:48.757655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:48.759710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:48.760694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 630/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Tue Oct 4 10:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:48.766170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:48.816898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:48.871696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2ad2b0-610d-41a2-9bb0-1bf716232278 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:48.910417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:48.911051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:48.911278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:48.911543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:48.912196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] RBAC: Adding query filter params (name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:48.912492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:48.913968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:48.923465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 606/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Tue Oct 4 10:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:48.929453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:49.030883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:49.050590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e3eddb-b38d-43fb-b4f2-930b2fecea0f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:49.088196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:49.088921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:49.089245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:49.089572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:49.102174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:49.103986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:49.120859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:49.121962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:49.123118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:49.123118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:49.123118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:49.123118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 631/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:10:49.128579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:49.246921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efde53d-b630-4f72-8c17-1af0529e780d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:49.257594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:49.284684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:49.285349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:49.285688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:49.286031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:49.286888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:49.287185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:49.288647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:49.303649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 607/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:10:49.310106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:49.430646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16a1dcc-b897-459e-9d51-4754a52b0eb9 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:49.467537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:49.468172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:49.468448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:49.468715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:49.471996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:49.478553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:49.480517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:10:49.493265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:49.494270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:49.495272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 632/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:49.500868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:49.609761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffc610-8cfb-48bd-934f-d3036803a623 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:49.646542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:49.647098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:49.647408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:49.647713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:49.648765 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:49.649229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:49.650773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:49.671057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 608/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Oct 4 10:10:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:49.677095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:49.786919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0deb56-9c0a-4513-86e3-a2e78f238c8b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:49.841397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:49.842586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:49.843202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:49.843638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:49.916330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:49.924062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:49.926402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:50.034228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 633/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:10:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:10:50.041156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:50.070553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:50.156058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17749ed1-8a9f-4aa2-83fd-7b8124b428f4 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:50.192487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:50.193163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:50.193432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:50.193781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:50.206798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:50.208393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:50.225651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:50.227500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:50.228408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:50.228408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:50.228408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:50.228408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 609/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:50.233467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:50.342177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:50.348530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de205ca-30e1-47cb-ad6b-9ee3836cb7d8 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:50.387553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:50.388403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:50.388680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:50.388950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:50.389824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:50.390216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:50.391993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:50.408019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 634/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:50.413909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:50.498591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:50.521611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db501d-2d17-4cee-8b3a-97ce5325302f None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:50.561020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:50.561716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:50.562034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:50.562384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:50.570878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:50.572857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:10:50.585642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:50.587241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:50.588309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 610/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:50.593915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:50.725763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12198be6-bbed-4d55-ab8c-1114595c392f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:50.764921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:50.765528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:50.765805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:50.766075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:50.766795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:50.767097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:50.768894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43bf15c-34ae-4ff1-ae16-c695486d3637 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:50.783643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 635/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:50.791330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:50.910229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ba1688-365b-4adf-bcfe-529ed2c115fb None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:50.945246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:50.946653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:50.946949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:50.947232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:50.948024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] RBAC: Adding query filter params (role.id=8b01b60bdf664e84ad21cc7e7d3ff607, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=c873c7e9014b46c8b1bf717f1e0fe514) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:50.948380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:50.949854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe79c4d-e4ea-45fc-8449-7557a8b7ec99 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:50.956052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 611/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:10:50 2022] GET /identity/v3/role_assignments?user.id=c873c7e9014b46c8b1bf717f1e0fe514&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=8b01b60bdf664e84ad21cc7e7d3ff607 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:53.244557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3515-fe6f-48dc-8cad-b0040279f554 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:53.245108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3515-fe6f-48dc-8cad-b0040279f554 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:53.245219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e3515-fe6f-48dc-8cad-b0040279f554 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:53.246995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 636/1247] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:10:53.252067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:53.252503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:53.252602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:53.259452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:53.271428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:10:53.272619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:10:53.341524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 612/1248] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:10:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:10:53.629091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3515-fe6f-48dc-8cad-b0040279f554 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:53.649200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:53.738957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e3515-fe6f-48dc-8cad-b0040279f554 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:53.778848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:53.779478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:53.779753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:53.780024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:53.790934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:10:53.796409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:53.797739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:53.798805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 637/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:10:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:53.807556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:53.908410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db669091-6845-4cf9-95cd-778f8eb8debe None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:53.942156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:53.948462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:53.949176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:53.949546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:53.949893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:53.950854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:53.951181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:53.952710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:53.961915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 613/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:10:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:53.967822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:54.053580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:54.079943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88802da8-c554-4947-8667-bf1d81ac387f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:54.121894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:54.122555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:54.122818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:54.123082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:54.134397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:54.135836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:54.151430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:54.152542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:54.153808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:54.153808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:54.153808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:54.153808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 638/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 04 10:10:54.158327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:54.284983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f12e65-692b-4f88-9fab-95a1d6f990af None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:54.321546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:54.322159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:54.322462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:54.322731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:54.323371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:54.323671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:54.325038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:54.337712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 614/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:54.343577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:54.447865 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f78da9-3fb2-4670-8a24-66c7149bc671 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:54.476080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:54.484643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:54.485276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:54.485550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:54.485819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:54.494621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:54.496082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:10:54.508784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:54.509827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:54.510842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 639/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:54.516049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:54.582769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:54.618288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5565b0f1-10d4-4f84-a682-ac87ed15e49c None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:54.654120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:54.654764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:54.655041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:54.655367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:54.656089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:54.656393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:54.657803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:54.675504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 615/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Oct 4 10:10:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:54.682110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:54.787177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aac4513-ff5f-4962-b20f-48dffa67cf8b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:54.824916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:54.825524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:54.825808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:54.826080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:54.880147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:54.893089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:54.894572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:54.988793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 640/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:10:54 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:10:54.994617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:55.090490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef1f748-9356-45f6-b3ec-53dfb4e5752d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:55.102825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:55.124085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:55.124697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:55.124972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:55.125242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:55.135561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:55.136938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:55.151850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:55.152977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:55.154062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:55.154062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:55.154062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:55.154062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 616/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:55.158763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:55.251980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:55.263967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6e2c-5391-42de-ac92-e35c6253b682 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:55.297400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:55.297995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:55.298299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:55.298571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:55.299207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:55.299507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:55.300862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:55.311610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 641/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:55.317398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:55.413175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e7a78-f331-4503-8324-f39165838050 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:55.445975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:55.446877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:55.447315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:55.447618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:55.454107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:55.455550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:10:55.467176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:55.468173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:10:55.469090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 617/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:55.474262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:55.571589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4848f-b18a-4d70-a4e0-104c46b73c4b None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:55.604412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:55.604997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:55.605273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:55.605542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:55.606241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:55.606545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:55.607959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d85642d-e97d-49a5-be3b-b69e1d944e63 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:55.620226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 642/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:10:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:55.626274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:55.650783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:55.722083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426d722a-59e2-4667-b4e2-fb8a9604d3e3 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:55.754356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:55.754938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:55.755214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:55.755481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:55.756185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=c873c7e9014b46c8b1bf717f1e0fe514) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:55.756488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:55.758000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00ca348-e75c-4044-9140-0fab7798a239 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:55.763136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 618/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:10:55 2022] GET /identity/v3/role_assignments?user.id=c873c7e9014b46c8b1bf717f1e0fe514&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=e8cffbbc9b164ec5b458727785767845 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:57.868251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe24b5-cd16-44d6-9cde-c405f908b15a None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:57.868786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe24b5-cd16-44d6-9cde-c405f908b15a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:57.868786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbe24b5-cd16-44d6-9cde-c405f908b15a None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:57.870123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 643/1261] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:10:57.875371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:57.875809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:57.875913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:57.894097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:10:57.895299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:10:57.958700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 619/1262] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:10:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:10:58.242444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe24b5-cd16-44d6-9cde-c405f908b15a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:58.255594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:58.347195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbe24b5-cd16-44d6-9cde-c405f908b15a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:58.379354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:58.379961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:58.380237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:58.380509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:58.390777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:10:58.395496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:58.396549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:58.397561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 644/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:10:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:58.402768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:58.420259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:58.500497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab60d093-778e-477d-8d64-8ad5e336e027 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:58.533780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:58.534409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:58.534687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:58.534958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:58.535607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:58.535906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:58.537252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:58.545700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 620/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:10:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:58.551627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:58.616342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:58.649642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa08c1c7-501e-4d77-9985-04df04346540 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:58.682717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:58.683328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:58.683598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:58.683862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:58.693238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:58.694704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:58.708422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:58.709509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:58.710595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:58.710595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:58.710595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:58.710595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 645/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:58.715911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:58.794278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:58.814340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4d32a9-1d24-4d45-a2af-2933eeb555a6 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:58.853421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:58.854035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:58.854415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:58.854719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:58.855361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:58.855662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:58.857036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:58.870735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 621/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:58.876427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:58.973465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34714b33-b2af-4f9a-91bf-19c44852be55 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:58.992757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:59.007008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.008219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.008494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.008764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.015943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.017372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:10:59.028182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:59.029288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:59.030437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 646/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:59.034960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:59.129409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8270eb17-d85a-475f-8edd-b06f5dd3d011 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:59.162282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.162871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.163147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.163417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.164176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:59.164474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.165878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.174127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:59.184905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 622/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Oct 4 10:10:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:59.191075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:59.291224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299d6a2-9010-4197-9b9e-1cf4fa654947 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:59.323248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.323866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.324142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.324413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.334443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.336796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:10:59.350308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:59.351440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:10:59.352563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:10:59.352563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:10:59.352563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:59.352563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 647/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:10:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:59.357198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:59.452129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa1d9c-c410-4100-9aef-17ca7f88744b None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:59.485715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.486343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.486626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.486899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.487529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:59.487830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.489175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.500570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 623/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:10:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:59.506049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:59.538340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:59.605604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ae7b3e-9958-496d-b15a-4a4038581f0c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:59.638962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.639579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.639855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.640126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.646681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.648223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:10:59.660065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:10:59.661109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:10:59.662255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 648/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:10:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:10:59.667099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:59.722372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:59.762385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf828b0e-a8dc-4185-ae83-cbd42d498460 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:59.793313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.793893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.794165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.794457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.795128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:10:59.795425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.796836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05001b04-64f9-402b-9a5a-049ac0213e77 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.808541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 624/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:10:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:10:59.814640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:10:59.902934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:10:59.912441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0944c2-1c66-4922-bddc-eb486f15d481 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:10:59.944931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60451b6-3712-49d1-901f-e9903c085475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:10:59.945651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60451b6-3712-49d1-901f-e9903c085475 None admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:10:59.945929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60451b6-3712-49d1-901f-e9903c085475 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:10:59.946213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60451b6-3712-49d1-901f-e9903c085475 None admin] PATH_INFO: `/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/c873c7e9014b46c8b1bf717f1e0fe514/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:10:59.948078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60451b6-3712-49d1-901f-e9903c085475 None admin] RBAC: Authorizing `identity:create_grant(project_id=2e15ff0840284da2afb5e33d5453289e, user_id=c873c7e9014b46c8b1bf717f1e0fe514, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:10:59.949495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60451b6-3712-49d1-901f-e9903c085475 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:10:59.962347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 649/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Tue Oct 4 10:10:59 2022] PUT /identity/v3/projects/2e15ff0840284da2afb5e33d5453289e/users/c873c7e9014b46c8b1bf717f1e0fe514/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:11:02.075390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ffd6e-5888-4f64-96e9-e052363207de None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:02.075897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ffd6e-5888-4f64-96e9-e052363207de None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:02.075897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ffd6e-5888-4f64-96e9-e052363207de None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:02.077181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 625/1274] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:11:02.082516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:02.082903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:02.083058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:02.102106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:11:02.103337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:11:02.167393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 650/1275] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:11:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:11:02.440079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ffd6e-5888-4f64-96e9-e052363207de None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:02.538936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ffd6e-5888-4f64-96e9-e052363207de None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:02.556140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:02.573773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:02.574422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:02.574701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:02.574973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:02.584656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 04 10:11:02.590282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:02.592646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 626/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Tue Oct 4 10:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:02.597332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:02.675224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:02.693707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368f0f91-97dd-4ba7-a4e9-c75276cfcfc8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:02.726595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:02.727179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:02.727454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:02.727730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:02.728380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:02.728679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:02.730021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:02.737707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 651/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Tue Oct 4 10:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:02.744481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:02.849423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5e82e1-7094-477a-8bfa-f1b7b5c3f963 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:02.883764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:02.884380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:02.884658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:02.884933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:02.894485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:02.895929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:11:02.910352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:02.911516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:11:02.912621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:11:02.912621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:11:02.912621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:02.912621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 627/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:02.917689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:03.013167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136f7af-befe-4371-8c13-0e61fbebe8af None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:03.044993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:03.045591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:03.045869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:03.046142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:03.046879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:03.047182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:03.048537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:03.060483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 652/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:03.068107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:03.097434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:03.165202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58a1a5-006a-4f45-9cca-abf4ad2ccfc9 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:03.199704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:03.200448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:03.200770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:03.201058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:03.209100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:03.210570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 04 10:11:03.221751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:03.222890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:03.224058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 628/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:11:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:03.229680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:03.233473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:03.326023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876bad13-7048-4784-a164-452a3e7f5c09 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:03.359646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:03.360230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:03.360502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:03.360770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:03.361480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:03.361777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:03.363214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:03.379885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 653/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Tue Oct 4 10:11:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:11:03.385890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:03.472694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:03.483340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab18df9e-e285-41bb-9044-aa0063c199fe None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:03.515437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:03.516062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:03.516349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:03.516625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:03.580835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:03.582195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:03.668351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:03.678396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 629/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Tue Oct 4 10:11:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 04 10:11:03.684150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:03.743377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:03.787169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e950b-caa2-40c0-8281-dc5ad2e98c5c None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:03.819163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:03.820449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:03.820744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:03.820970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:03.830626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:03.832094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 04 10:11:03.846741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:03.847804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 04 10:11:03.849838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 04 10:11:03.849838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 04 10:11:03.849838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:11:03.849838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 654/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:03.854475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:03.947812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be81225-5373-40dd-a07d-dc04ce55075d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:03.980106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:03.980693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:03.980965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:03.981229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:03.981846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:03.982140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:03.983509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:03.994361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 630/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Tue Oct 4 10:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:03.999961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:04.097564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e756a6c-bfee-4ea8-ba42-b11a9b8e544a None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:04.111210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:04.132672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:04.133287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:04.133561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:04.133827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:04.140187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:04.141610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 04 10:11:04.153616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:04.154688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: ERROR keystone.server.flask.application Oct 04 10:11:04.155755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 655/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Tue Oct 4 10:11:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:04.160526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:04.208990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:04.257604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971c0a6e-b267-4126-b3a0-c8be71b8df2d None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:04.289652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:04.290254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:04.290529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:04.290797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:04.291466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:04.291764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:04.293155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958032ea-91ed-4434-ae8a-b19012fa1145 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:04.304569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 631/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Tue Oct 4 10:11:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:04.310716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:04.406628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a8f1ae-cf2c-45bc-a5d9-8fa0bd2b518f None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:04.439039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:04.439633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:04.439910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:04.440182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:04.440897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] RBAC: Adding query filter params (role.id=e8cffbbc9b164ec5b458727785767845, scope.project.id=2e15ff0840284da2afb5e33d5453289e, user.id=c873c7e9014b46c8b1bf717f1e0fe514) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:04.441205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:04.442672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddda65ca-b697-4513-a493-6d8a8c98bd19 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:04.448800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 656/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Tue Oct 4 10:11:04 2022] GET /identity/v3/role_assignments?user.id=c873c7e9014b46c8b1bf717f1e0fe514&scope.project.id=2e15ff0840284da2afb5e33d5453289e&role.id=e8cffbbc9b164ec5b458727785767845 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:06.595624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946a62-7a89-446b-bd65-4565d7fafb56 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:06.596184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946a62-7a89-446b-bd65-4565d7fafb56 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:06.596184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd946a62-7a89-446b-bd65-4565d7fafb56 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:06.597484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 632/1288] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:11:06.602707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:06.603088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:06.603245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:06.621779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:11:06.623033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:11:06.652211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:06.688174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 657/1289] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:11:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:11:06.958615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd946a62-7a89-446b-bd65-4565d7fafb56 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:07.054619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd946a62-7a89-446b-bd65-4565d7fafb56 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:07.064582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:07.086827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:07.087432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:07.087705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:07.087973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] PATH_INFO: `/v3/services/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:07.088534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:07.089866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:07.098190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:07.099430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:07.100287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 633/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:11:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 141 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:07.105702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:07.200872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a9d651-1c51-43a7-9f17-cdd4ec1d365b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:07.242760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:07.243362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:07.243707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:07.244039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:07.244709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:07.245041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:07.246619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b85eec7-9bcb-414b-85dd-140758655c06 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:07.252554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 658/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Oct 4 10:11:07 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:07.258744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:07.358463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c993c8-53e3-407f-8806-2d425bf10e4e None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:07.389865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:07.390480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:07.390756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:07.391027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] PATH_INFO: `/v3/services` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:07.391656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:07.391932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:07.393284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e43ca4-8cea-429c-b31b-0d85d6bca501 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:07.397440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 634/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Oct 4 10:11:07 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:09.530739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a34db-e530-482a-b133-f0feec5e34fd None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:09.531523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a34db-e530-482a-b133-f0feec5e34fd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:09.531523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0a34db-e530-482a-b133-f0feec5e34fd None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:09.532674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 659/1293] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:11:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:11:09.539710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4b9a6-1957-4a9b-937b-ee6a51524b72 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:09.540135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4b9a6-1957-4a9b-937b-ee6a51524b72 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:09.540413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe4b9a6-1957-4a9b-937b-ee6a51524b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:09.558858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5fe4b9a6-1957-4a9b-937b-ee6a51524b72 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:11:09.560070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5fe4b9a6-1957-4a9b-937b-ee6a51524b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:11:09.595015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:09.634163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 635/1294] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:11:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:11:09.911724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a34db-e530-482a-b133-f0feec5e34fd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:09.943030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:10.011751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0a34db-e530-482a-b133-f0feec5e34fd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:10.045209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d659291c-ff10-492c-9a36-0e58c31f2636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:10.045807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659291c-ff10-492c-9a36-0e58c31f2636 None admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:10.046086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659291c-ff10-492c-9a36-0e58c31f2636 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:10.046385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d659291c-ff10-492c-9a36-0e58c31f2636 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:10.047033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659291c-ff10-492c-9a36-0e58c31f2636 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:10.048400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d659291c-ff10-492c-9a36-0e58c31f2636 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:10.062696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 660/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Tue Oct 4 10:11:09 2022] POST /identity/v3/services => generated 246 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:11:12.215157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3075-7a2d-44ea-84b4-33ff3abc859c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:12.215710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3075-7a2d-44ea-84b4-33ff3abc859c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:12.215888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ca3075-7a2d-44ea-84b4-33ff3abc859c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:12.217322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 636/1296] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:11:12.222784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:12.223192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:12.223388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:12.241982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:11:12.243195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:11:12.315754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 661/1297] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:11:12 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:11:12.583967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca3075-7a2d-44ea-84b4-33ff3abc859c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:12.682238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ca3075-7a2d-44ea-84b4-33ff3abc859c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:12.716430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:12.717072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:12.717346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:12.717616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:12.718190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:12.719568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:12.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:12.730061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:12.731105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 637/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:11:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:12.735985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:12.753434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:12.832466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace08590-a7bd-403c-80bb-565adecef5c6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:12.869165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:12.869764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:12.870043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:12.870344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:12.870971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:12.871273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:12.872627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:12.877562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 662/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Oct 4 10:11:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:12.884101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:12.914389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:12.979733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7137570a-e67f-46b5-b53f-f071bf101cf1 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:13.011374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:13.011954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:13.012227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:13.012498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:13.013177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] RBAC: Adding query filter params (interface=public, service_id=63fc8478e9e44259b98acfcf5da0d624, region_id=RegionOne) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:13.013477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:13.014843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03de6eb-0c63-49e1-af5a-c31a207b0e63 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:13.019573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 638/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Tue Oct 4 10:11:12 2022] GET /identity/v3/endpoints?service_id=63fc8478e9e44259b98acfcf5da0d624&interface=public®ion_id=RegionOne => generated 194 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:13.025142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:13.107673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:13.125709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59eea24-8d76-44c9-a1b0-ee1333816de6 None admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:13.158160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d4a973-3f79-4d5b-87ec-6def3592be46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:13.158845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4a973-3f79-4d5b-87ec-6def3592be46 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:13.159120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4a973-3f79-4d5b-87ec-6def3592be46 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:13.159390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d4a973-3f79-4d5b-87ec-6def3592be46 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:13.159988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4a973-3f79-4d5b-87ec-6def3592be46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:13.161334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d4a973-3f79-4d5b-87ec-6def3592be46 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:13.168587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 663/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Tue Oct 4 10:11:13 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:11:15.352779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee949da-b90a-477d-ab8a-ca65ba16534f None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:15.353303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee949da-b90a-477d-ab8a-ca65ba16534f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:15.353303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee949da-b90a-477d-ab8a-ca65ba16534f None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:15.354698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 639/1302] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:11:15.359869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:15.360299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:15.360419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:15.379780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:11:15.380976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:11:15.450404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 664/1303] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:11:15 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 04 10:11:15.731147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee949da-b90a-477d-ab8a-ca65ba16534f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:15.827083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:15.834080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee949da-b90a-477d-ab8a-ca65ba16534f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:15.867500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:15.868114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:15.868386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:15.868651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:15.869214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:15.870583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 04 10:11:15.879030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 04 10:11:15.880203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.get_or_create( Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application with Lock( Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self._enter() Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application return self.creator() Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application created_value = creator( Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: ERROR keystone.server.flask.application Oct 04 10:11:15.881517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 640/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Tue Oct 4 10:11:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 04 10:11:15.886153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:15.958566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:15.983747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09dfb984-9eb4-419b-948e-f7eb8dac3250 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:16.016841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:16.017446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:16.017724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:16.017995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] PATH_INFO: `/v3/services` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:16.018655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:11:16.018955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:16.020332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f9d7d-bc1d-4176-b8df-3c1de7af7936 None admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:16.025466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 665/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Tue Oct 4 10:11:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:11:16.031201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:11:16.127931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ae3af6-f92e-4b59-8b5e-d973cc6a3751 None admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:11:16.160141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975f1a09-ffcf-4a68-a7e1-992ad2d4aaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:11:16.160656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1a09-ffcf-4a68-a7e1-992ad2d4aaab None admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:11:16.160928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1a09-ffcf-4a68-a7e1-992ad2d4aaab None admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:11:16.161195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975f1a09-ffcf-4a68-a7e1-992ad2d4aaab None admin] PATH_INFO: `/v3/endpoints` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:11:16.161840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f1a09-ffcf-4a68-a7e1-992ad2d4aaab None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:11:16.163858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975f1a09-ffcf-4a68-a7e1-992ad2d4aaab None admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:11:16.183459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 641/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:11:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:11:26.022787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=30 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:11:26.180671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:21.640033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5696ae2-373d-42b8-a15c-1581b1aea988 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:21.641025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5696ae2-373d-42b8-a15c-1581b1aea988 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:21.641549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5696ae2-373d-42b8-a15c-1581b1aea988 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:21.648083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 666/1307] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:21 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:21.649416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35245a-63ee-471c-971b-d5f70692a7dc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:21.649904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35245a-63ee-471c-971b-d5f70692a7dc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:21.650087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35245a-63ee-471c-971b-d5f70692a7dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:21.754388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9f35245a-63ee-471c-971b-d5f70692a7dc None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:21.755989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9f35245a-63ee-471c-971b-d5f70692a7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:21.880532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 642/1308] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:12:22.178941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c384110-0015-4d35-bc4c-48bd87f5dd72 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:22.179219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c384110-0015-4d35-bc4c-48bd87f5dd72 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:22.179681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c384110-0015-4d35-bc4c-48bd87f5dd72 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:22.181043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 667/1309] 10.222.0.13 () {54 vars in 964 bytes} [Tue Oct 4 10:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:22.187927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd336f-6a17-498b-89ed-cf5c716fcad4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:22.188205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd336f-6a17-498b-89ed-cf5c716fcad4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:22.188610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd336f-6a17-498b-89ed-cf5c716fcad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:22.279807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d1fd336f-6a17-498b-89ed-cf5c716fcad4 None None] MFA Rules not processed for user `0e568115c67243d3904359b84e0a790e`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:22.281164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d1fd336f-6a17-498b-89ed-cf5c716fcad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:22.306300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:22.367856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 643/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Tue Oct 4 10:12:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:22.376821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c384110-0015-4d35-bc4c-48bd87f5dd72 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:22.379853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2c384110-0015-4d35-bc4c-48bd87f5dd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:22.495621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c384110-0015-4d35-bc4c-48bd87f5dd72 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:22.574861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:22.575733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:22.576005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:22.576272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:22.577857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:22.657118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:22.714694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:22.716144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d805176-93ca-4061-acfd-bb531f794b85 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:22.735624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 668/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:25.211054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec661a5d-6097-4e05-b221-3f5ea618fcfd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:25.211581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec661a5d-6097-4e05-b221-3f5ea618fcfd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:25.211581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec661a5d-6097-4e05-b221-3f5ea618fcfd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:25.212881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 644/1312] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:25.218219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f00e1-7434-46f2-94a8-8277ebe22212 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:25.218590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f00e1-7434-46f2-94a8-8277ebe22212 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:25.218765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f00e1-7434-46f2-94a8-8277ebe22212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:25.289968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b68f00e1-7434-46f2-94a8-8277ebe22212 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:25.291230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b68f00e1-7434-46f2-94a8-8277ebe22212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:25.385525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 669/1313] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:25.668744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec661a5d-6097-4e05-b221-3f5ea618fcfd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:25.670316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ec661a5d-6097-4e05-b221-3f5ea618fcfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:25.773841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec661a5d-6097-4e05-b221-3f5ea618fcfd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:25.848532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:25.850875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:25.851696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:25.852027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:25.852436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:25.854256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:25.960337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:25.961677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5423ae-9176-4b2d-832e-f8b34e51624a service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:25.979771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 645/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:28.509459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf7c3f-fa66-4435-8e56-7fc9717aa3d1 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:28.510024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf7c3f-fa66-4435-8e56-7fc9717aa3d1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:28.510024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacf7c3f-fa66-4435-8e56-7fc9717aa3d1 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:28.511365 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 670/1315] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:28.516608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c81cbc-a007-4495-8be2-e9bf9d545ad0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:28.517039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c81cbc-a007-4495-8be2-e9bf9d545ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:28.517134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c81cbc-a007-4495-8be2-e9bf9d545ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:28.582230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-51c81cbc-a007-4495-8be2-e9bf9d545ad0 None None] MFA Rules not processed for user `e55c08d9c6404546b2c05b48636468bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:28.583454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-51c81cbc-a007-4495-8be2-e9bf9d545ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:28.634065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:28.678269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 646/1316] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:28 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:28.961508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf7c3f-fa66-4435-8e56-7fc9717aa3d1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:28.963062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bacf7c3f-fa66-4435-8e56-7fc9717aa3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:28.973511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:29.064514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacf7c3f-fa66-4435-8e56-7fc9717aa3d1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:29.136478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:29.137066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:29.137340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:29.137610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:29.139213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:29.201490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:29.248327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:29.249792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac4411-81b0-4e2b-804a-b0feb42badda service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:29.267332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 671/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:28 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:32.013981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6ea9e-9c27-4e0d-bbcc-0801874fde39 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:32.014560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6ea9e-9c27-4e0d-bbcc-0801874fde39 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:32.014560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6ea9e-9c27-4e0d-bbcc-0801874fde39 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:32.015832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 647/1318] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:32.021445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c8d5e-3e6f-4529-89cc-73d1f493beef None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:32.021864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c8d5e-3e6f-4529-89cc-73d1f493beef None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:32.021965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157c8d5e-3e6f-4529-89cc-73d1f493beef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:32.099254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-157c8d5e-3e6f-4529-89cc-73d1f493beef None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:32.100446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-157c8d5e-3e6f-4529-89cc-73d1f493beef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:32.192822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 672/1319] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:32.484690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6ea9e-9c27-4e0d-bbcc-0801874fde39 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:32.486275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c7e6ea9e-9c27-4e0d-bbcc-0801874fde39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:32.586455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6ea9e-9c27-4e0d-bbcc-0801874fde39 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:32.658314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:32.658908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:32.659178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:32.659446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:32.661052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:32.667744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:32.767318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:32.769180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bc70c0-27ec-4898-b421-f4b1d54ee44c service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:32.786757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 648/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:32 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:35.622899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c02a73-b9c9-410f-b0eb-2c59da114415 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:35.623461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c02a73-b9c9-410f-b0eb-2c59da114415 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:35.623461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c02a73-b9c9-410f-b0eb-2c59da114415 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:35.624799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 673/1321] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:35.631158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e63c2-ad59-4cf2-b5dc-73e195e3cbd5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:35.631725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e63c2-ad59-4cf2-b5dc-73e195e3cbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:35.631725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0e63c2-ad59-4cf2-b5dc-73e195e3cbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:35.701563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1c0e63c2-ad59-4cf2-b5dc-73e195e3cbd5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:35.702809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1c0e63c2-ad59-4cf2-b5dc-73e195e3cbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:35.764668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:35.806926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 649/1322] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:36.090054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c02a73-b9c9-410f-b0eb-2c59da114415 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:36.091668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-24c02a73-b9c9-410f-b0eb-2c59da114415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:36.111294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:36.194013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c02a73-b9c9-410f-b0eb-2c59da114415 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:36.266433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:36.267029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:36.267301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:36.267566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:36.270004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:36.341538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:36.380325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:36.381752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c94355-7cc4-4485-969f-19b87d8bf6c2 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:36.399529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 674/1323] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:38.633764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e0d47-a0c0-4e3c-b058-c84ae61cb83e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:38.634155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e0d47-a0c0-4e3c-b058-c84ae61cb83e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:38.634469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092e0d47-a0c0-4e3c-b058-c84ae61cb83e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:38.636040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 650/1324] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:38.641150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd7216-7efb-4652-be9c-2bc034c461f0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:38.641570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd7216-7efb-4652-be9c-2bc034c461f0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:38.641681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd7216-7efb-4652-be9c-2bc034c461f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:38.716856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5ffd7216-7efb-4652-be9c-2bc034c461f0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:38.718070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5ffd7216-7efb-4652-be9c-2bc034c461f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:38.816457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 675/1325] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:39.100931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e0d47-a0c0-4e3c-b058-c84ae61cb83e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:39.102496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-092e0d47-a0c0-4e3c-b058-c84ae61cb83e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:39.204848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092e0d47-a0c0-4e3c-b058-c84ae61cb83e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:39.277473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:39.278076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:39.278386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:39.278657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:39.280259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:39.296702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:39.394049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:39.396259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbc502c-625d-4bd5-9cea-b59521d91ced service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:39.416367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 651/1326] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:41.650384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60684fe8-f509-4664-9f27-ed7d28039011 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:41.651284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60684fe8-f509-4664-9f27-ed7d28039011 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:41.651559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60684fe8-f509-4664-9f27-ed7d28039011 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:41.652952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 676/1327] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:41.659125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97935620-58ee-4302-a246-82d2c80b3668 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:41.659561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97935620-58ee-4302-a246-82d2c80b3668 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:41.659813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97935620-58ee-4302-a246-82d2c80b3668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:41.726945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-97935620-58ee-4302-a246-82d2c80b3668 None None] MFA Rules not processed for user `e55c08d9c6404546b2c05b48636468bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:41.728142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-97935620-58ee-4302-a246-82d2c80b3668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:41.792715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:41.824211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 652/1328] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:41 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:42.108256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60684fe8-f509-4664-9f27-ed7d28039011 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:42.109807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-60684fe8-f509-4664-9f27-ed7d28039011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:42.141823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:42.217604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60684fe8-f509-4664-9f27-ed7d28039011 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:42.292931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:42.293524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:42.293794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:42.294058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:42.295666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:42.372984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:42.405503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:42.407031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05faa6-5692-459e-be99-4143a0a90b59 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:42.427296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 677/1329] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:44.901375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55401853-7942-45db-8448-8c676c49469e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:44.901902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55401853-7942-45db-8448-8c676c49469e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:44.901902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55401853-7942-45db-8448-8c676c49469e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:44.903363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 653/1330] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:44.909104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7357c-e5fb-4606-bff0-099ae56a4bd1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:44.909697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7357c-e5fb-4606-bff0-099ae56a4bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:44.909697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be7357c-e5fb-4606-bff0-099ae56a4bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:44.977195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9be7357c-e5fb-4606-bff0-099ae56a4bd1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:44.977911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9be7357c-e5fb-4606-bff0-099ae56a4bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:45.076660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 678/1331] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:45.376386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55401853-7942-45db-8448-8c676c49469e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:45.378588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-55401853-7942-45db-8448-8c676c49469e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:45.498065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55401853-7942-45db-8448-8c676c49469e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:45.570627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:45.571231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:45.571505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:45.571775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:45.573361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:45.600052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:45.682480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:45.683914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88e62f-348f-48c1-8ae1-9b1069b3f24b service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:45.702718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 654/1332] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:47.914826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486e3a-5076-4052-bf54-0320a0075cb6 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:47.915331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486e3a-5076-4052-bf54-0320a0075cb6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:47.915331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a486e3a-5076-4052-bf54-0320a0075cb6 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:47.916640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 679/1333] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:47.921820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce528869-f706-46a4-a96b-3a7cf542abab None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:47.922260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce528869-f706-46a4-a96b-3a7cf542abab None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:47.922397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce528869-f706-46a4-a96b-3a7cf542abab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:47.987394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ce528869-f706-46a4-a96b-3a7cf542abab None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:47.988586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ce528869-f706-46a4-a96b-3a7cf542abab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:48.062510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:48.086031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 655/1334] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:48.384471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486e3a-5076-4052-bf54-0320a0075cb6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:48.386028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a486e3a-5076-4052-bf54-0320a0075cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:48.427878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:48.492191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a486e3a-5076-4052-bf54-0320a0075cb6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:48.564993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:48.565589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:48.565867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:48.566136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:48.567758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:48.653276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:48.679832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:48.681276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81469c5-2d30-4b8a-ba00-20a89c414f80 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:48.700083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 680/1335] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:51.206910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1f2f-1834-4b93-aa2a-43a5522c777b None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:51.207426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1f2f-1834-4b93-aa2a-43a5522c777b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:51.207426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7f1f2f-1834-4b93-aa2a-43a5522c777b None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:51.208723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 656/1336] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:51.215087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7714-0bb6-4a7d-b968-413dc21934b8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:51.215578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7714-0bb6-4a7d-b968-413dc21934b8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:51.215913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcc7714-0bb6-4a7d-b968-413dc21934b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:51.289808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8fcc7714-0bb6-4a7d-b968-413dc21934b8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:51.291060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8fcc7714-0bb6-4a7d-b968-413dc21934b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:51.388412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 681/1337] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:51.687562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e06a-0682-4f1f-9e4e-15b7c45130bb None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:51.687846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e06a-0682-4f1f-9e4e-15b7c45130bb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:51.688118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d4e06a-0682-4f1f-9e4e-15b7c45130bb None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:51.690167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 657/1338] 10.222.0.13 () {54 vars in 964 bytes} [Tue Oct 4 10:12:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:51.696479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8f975-8562-40aa-87e4-05192c75cf71 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:51.696758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8f975-8562-40aa-87e4-05192c75cf71 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:51.697023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff8f975-8562-40aa-87e4-05192c75cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:51.750916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:51.772536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bff8f975-8562-40aa-87e4-05192c75cf71 None None] MFA Rules not processed for user `0e568115c67243d3904359b84e0a790e`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:51.773720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bff8f975-8562-40aa-87e4-05192c75cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:51.847828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 682/1339] 10.222.0.13 () {58 vars in 1061 bytes} [Tue Oct 4 10:12:51 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:12:51.856687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e06a-0682-4f1f-9e4e-15b7c45130bb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:51.858172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c4d4e06a-0682-4f1f-9e4e-15b7c45130bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:51.958734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d4e06a-0682-4f1f-9e4e-15b7c45130bb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:52.035924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:52.036520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:52.036793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:52.037063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:52.038714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:52.073983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:52.147608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:52.149026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09ac97b-ca35-42cd-b6e5-95998b4dd5e7 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:52.167294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 658/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:12:56.799349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8b577-26e8-4e7a-94a6-cb5c4e9c06fa None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:56.799935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8b577-26e8-4e7a-94a6-cb5c4e9c06fa None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:56.799935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8b577-26e8-4e7a-94a6-cb5c4e9c06fa None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:56.801068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 683/1341] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:12:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:12:56.806451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8e621-4aa2-43ca-a664-2f2958524013 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:56.806887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8e621-4aa2-43ca-a664-2f2958524013 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:56.807007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef8e621-4aa2-43ca-a664-2f2958524013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:56.874812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9ef8e621-4aa2-43ca-a664-2f2958524013 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:12:56.875952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9ef8e621-4aa2-43ca-a664-2f2958524013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:56.963669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:56.976079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 659/1342] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:12:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:12:57.259046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8b577-26e8-4e7a-94a6-cb5c4e9c06fa None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:12:57.260574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-86c8b577-26e8-4e7a-94a6-cb5c4e9c06fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:57.365978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8b577-26e8-4e7a-94a6-cb5c4e9c06fa None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:12:57.403050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:12:57.438594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:12:57.439222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:12:57.439570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:12:57.439841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:12:57.441415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:12:57.552360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:12:57.553723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477fa2a8-0813-492e-bc32-c2052614c038 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:12:57.571900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 684/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:12:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:00.386905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816fc1-b30a-45b3-ade9-9f82a592b820 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:00.387461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816fc1-b30a-45b3-ade9-9f82a592b820 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:00.387461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28816fc1-b30a-45b3-ade9-9f82a592b820 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:00.388942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 660/1344] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:00.394484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd87a-8766-4c26-956e-6f77681e04c5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:00.394897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd87a-8766-4c26-956e-6f77681e04c5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:00.395031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcd87a-8766-4c26-956e-6f77681e04c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:00.460047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f5bcd87a-8766-4c26-956e-6f77681e04c5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:00.461229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f5bcd87a-8766-4c26-956e-6f77681e04c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:00.479377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:00.560631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 685/1345] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:00.845927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816fc1-b30a-45b3-ade9-9f82a592b820 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:00.847584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-28816fc1-b30a-45b3-ade9-9f82a592b820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:00.959079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28816fc1-b30a-45b3-ade9-9f82a592b820 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:01.032268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:01.032868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:01.033144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:01.033406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:01.034999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:01.079876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:01.150405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:01.152462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5519955-0e74-41f8-bc9c-1b426071b87e service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:01.171314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 661/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:05.276900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c53b-f35e-4fcd-81d8-1ab10901ac69 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:05.277411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c53b-f35e-4fcd-81d8-1ab10901ac69 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:05.277488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a4c53b-f35e-4fcd-81d8-1ab10901ac69 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:05.278832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 686/1347] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:13:05.284799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ca725-89a5-42ab-956b-fb98f32733c3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:05.285294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ca725-89a5-42ab-956b-fb98f32733c3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:05.285371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ca725-89a5-42ab-956b-fb98f32733c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:05.366506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-520ca725-89a5-42ab-956b-fb98f32733c3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:05.367804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-520ca725-89a5-42ab-956b-fb98f32733c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:05.471343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:05.473544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 662/1348] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:05.754678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4c53b-f35e-4fcd-81d8-1ab10901ac69 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:05.756206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a4a4c53b-f35e-4fcd-81d8-1ab10901ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:05.857924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a4c53b-f35e-4fcd-81d8-1ab10901ac69 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:05.906282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:05.931638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:05.932229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:05.932503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:05.932772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:05.934388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:06.040772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:06.042135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8732767-03f5-43a2-a50c-e58023cfbff9 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:06.060141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 687/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:11.008771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c25b0b-3bea-4823-bff1-0ac53413d719 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:11.009274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c25b0b-3bea-4823-bff1-0ac53413d719 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:11.009352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c25b0b-3bea-4823-bff1-0ac53413d719 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:11.010718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 663/1350] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:13:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:11.015793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017db22-31b6-43d2-92b4-95e6c7f1d4df None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:11.017451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017db22-31b6-43d2-92b4-95e6c7f1d4df None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:11.018483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d017db22-31b6-43d2-92b4-95e6c7f1d4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:11.130630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d017db22-31b6-43d2-92b4-95e6c7f1d4df None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:11.133312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d017db22-31b6-43d2-92b4-95e6c7f1d4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:11.159105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:11.293282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 688/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:13:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:15.716120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8c4c6-8e75-4f08-b96d-1f9f6a1d41e2 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:15.716679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8c4c6-8e75-4f08-b96d-1f9f6a1d41e2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:15.716907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b8c4c6-8e75-4f08-b96d-1f9f6a1d41e2 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:15.718574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 664/1352] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:15.724150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3186-dd5d-4cec-b19e-1d5f924f8169 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:15.724603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3186-dd5d-4cec-b19e-1d5f924f8169 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:15.724708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8b3186-dd5d-4cec-b19e-1d5f924f8169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:15.804540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5f8b3186-dd5d-4cec-b19e-1d5f924f8169 None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:15.805734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5f8b3186-dd5d-4cec-b19e-1d5f924f8169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:15.881996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:15.920335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 689/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:13:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:17.268094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022048e-7000-432a-aa03-bfc240006dc6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:17.268616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022048e-7000-432a-aa03-bfc240006dc6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:17.268861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c022048e-7000-432a-aa03-bfc240006dc6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:17.270267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 665/1354] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:17.275335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982471-e4a0-4712-804f-82cc072bd7ca None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:17.275776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982471-e4a0-4712-804f-82cc072bd7ca None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:17.275871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982471-e4a0-4712-804f-82cc072bd7ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:17.353349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-db982471-e4a0-4712-804f-82cc072bd7ca None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:17.354581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-db982471-e4a0-4712-804f-82cc072bd7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:17.461268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 690/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:13:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:20.954053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071a202-bcde-43cf-8dd2-58daeb2436c7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:20.954671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071a202-bcde-43cf-8dd2-58daeb2436c7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:20.954671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5071a202-bcde-43cf-8dd2-58daeb2436c7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:20.956200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 666/1356] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:20.962397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e8a1-3c7b-435f-8d96-81ee246dda0a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:20.963299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e8a1-3c7b-435f-8d96-81ee246dda0a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:20.963816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5e8a1-3c7b-435f-8d96-81ee246dda0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:21.008277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:21.046325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bdf5e8a1-3c7b-435f-8d96-81ee246dda0a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:21.047584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bdf5e8a1-3c7b-435f-8d96-81ee246dda0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:21.155991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 691/1357] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:21.457966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d6a7b-a9c8-418d-87a2-8ac387dd836a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:21.458295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d6a7b-a9c8-418d-87a2-8ac387dd836a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:21.458640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81d6a7b-a9c8-418d-87a2-8ac387dd836a None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:21.464026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 667/1358] 10.222.0.13 () {54 vars in 961 bytes} [Tue Oct 4 10:13:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:21.471894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3a304-6238-4661-b327-f4e0a2b46341 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:21.472178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3a304-6238-4661-b327-f4e0a2b46341 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:21.472453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f3a304-6238-4661-b327-f4e0a2b46341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:21.561355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d3f3a304-6238-4661-b327-f4e0a2b46341 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:21.562640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d3f3a304-6238-4661-b327-f4e0a2b46341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:21.615314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:21.702333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 692/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Oct 4 10:13:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:21.711513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d6a7b-a9c8-418d-87a2-8ac387dd836a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:21.713217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b81d6a7b-a9c8-418d-87a2-8ac387dd836a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:21.872816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81d6a7b-a9c8-418d-87a2-8ac387dd836a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:21.951492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:21.994922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:21.995522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:21.995792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:21.996066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:21.997654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:22.095613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:22.097216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb9944f-6873-4266-ba0c-b2b365539132 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:22.119777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 668/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:24.542298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e7b62-aa36-424d-a3ec-63beb6c61f9a None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:24.542861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e7b62-aa36-424d-a3ec-63beb6c61f9a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:24.542861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e7b62-aa36-424d-a3ec-63beb6c61f9a None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:24.544177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 693/1361] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:24.549616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c30fc-f4d5-4b2f-b834-8d404500adbd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:24.550054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c30fc-f4d5-4b2f-b834-8d404500adbd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:24.550144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6c30fc-f4d5-4b2f-b834-8d404500adbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:24.646083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1e6c30fc-f4d5-4b2f-b834-8d404500adbd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:24.647335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1e6c30fc-f4d5-4b2f-b834-8d404500adbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:24.686370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:24.754608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 669/1362] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:25.076318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e7b62-aa36-424d-a3ec-63beb6c61f9a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:25.077945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b83e7b62-aa36-424d-a3ec-63beb6c61f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:25.246394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e7b62-aa36-424d-a3ec-63beb6c61f9a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:25.259016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:25.401953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:25.402957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:25.403378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:25.403790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:25.406899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:25.506152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:25.507717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddbadb8-118f-40b0-8986-dfcb104f4038 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:25.526490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 694/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:28.024479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924871b-5ee6-4384-9870-66ded93428f1 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:28.025055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924871b-5ee6-4384-9870-66ded93428f1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:28.025055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c924871b-5ee6-4384-9870-66ded93428f1 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:28.026292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 670/1364] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:28.032357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017f846-262d-4bbb-8180-9e076b05a9f6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:28.032783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017f846-262d-4bbb-8180-9e076b05a9f6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:28.032891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017f846-262d-4bbb-8180-9e076b05a9f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:28.048261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:28.101161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c017f846-262d-4bbb-8180-9e076b05a9f6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:28.102470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c017f846-262d-4bbb-8180-9e076b05a9f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:28.207784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 695/1365] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:28.503201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb713ca8-d27b-47a9-b3de-d3075d475f93 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:28.503479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb713ca8-d27b-47a9-b3de-d3075d475f93 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:28.503754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb713ca8-d27b-47a9-b3de-d3075d475f93 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:28.505116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 671/1366] 10.222.0.13 () {54 vars in 961 bytes} [Tue Oct 4 10:13:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:28.511566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f98f-093f-4016-a5d6-8c4d531de0f5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:28.512167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f98f-093f-4016-a5d6-8c4d531de0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:28.512476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707f98f-093f-4016-a5d6-8c4d531de0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:28.596775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2707f98f-093f-4016-a5d6-8c4d531de0f5 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:28.597993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2707f98f-093f-4016-a5d6-8c4d531de0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:28.640867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:28.732738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 696/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Oct 4 10:13:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:28.741780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb713ca8-d27b-47a9-b3de-d3075d475f93 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:28.743691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fb713ca8-d27b-47a9-b3de-d3075d475f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:28.897419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb713ca8-d27b-47a9-b3de-d3075d475f93 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:28.920702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:29.023530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:29.024209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:29.024590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:29.025119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:29.026739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:29.122390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:29.124149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860845a2-82b0-4c58-8d83-669361e77c03 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:29.144175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 672/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:29.280789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205c179-5aa7-47a6-8153-f0ef8ea0577d None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:29.281886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205c179-5aa7-47a6-8153-f0ef8ea0577d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:29.282426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205c179-5aa7-47a6-8153-f0ef8ea0577d None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:29.284735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 697/1369] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:29.290808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbb0f8-e394-4fdc-b082-376779c00826 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:29.291230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbb0f8-e394-4fdc-b082-376779c00826 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:29.291606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbb0f8-e394-4fdc-b082-376779c00826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:29.333163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:29.379085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-40bbb0f8-e394-4fdc-b082-376779c00826 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:29.380575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40bbb0f8-e394-4fdc-b082-376779c00826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:29.502256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 673/1370] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:29.833657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7f540-40e8-4369-8cf0-4386ffc18e82 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:29.833657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7f540-40e8-4369-8cf0-4386ffc18e82 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:29.833657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7f540-40e8-4369-8cf0-4386ffc18e82 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:29.835608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 698/1371] 10.222.0.13 () {54 vars in 963 bytes} [Tue Oct 4 10:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:29.839123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9863-37d0-46e7-915b-7b5166c941f9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:29.839408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9863-37d0-46e7-915b-7b5166c941f9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:29.839680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd9863-37d0-46e7-915b-7b5166c941f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:29.920351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-77cd9863-37d0-46e7-915b-7b5166c941f9 None None] MFA Rules not processed for user `99de0be14e684b3696a4570987119ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:29.921678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-77cd9863-37d0-46e7-915b-7b5166c941f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:29.973588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:30.005189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 674/1372] 10.222.0.13 () {58 vars in 1060 bytes} [Tue Oct 4 10:13:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:30.012770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7f540-40e8-4369-8cf0-4386ffc18e82 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:30.014416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c3b7f540-40e8-4369-8cf0-4386ffc18e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:30.129415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7f540-40e8-4369-8cf0-4386ffc18e82 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:30.176695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:30.216255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:30.216861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:30.217133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:30.217397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:30.219777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:30.404101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:30.405550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccfb41c-68c5-4670-b8b4-ce805e768e1d service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:30.430755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 699/1373] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:31.450499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ecfc6-f706-46b0-9012-dcea26beaedc None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:31.451062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ecfc6-f706-46b0-9012-dcea26beaedc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:31.451062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ecfc6-f706-46b0-9012-dcea26beaedc None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:31.452506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 675/1374] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:31.458646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16007e0b-56c6-4783-bc02-ee34d803f6f1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:31.459072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16007e0b-56c6-4783-bc02-ee34d803f6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:31.459233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16007e0b-56c6-4783-bc02-ee34d803f6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:31.529419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-16007e0b-56c6-4783-bc02-ee34d803f6f1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:31.530690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-16007e0b-56c6-4783-bc02-ee34d803f6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:31.553587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:31.634243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 700/1375] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:13:31.928118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ecfc6-f706-46b0-9012-dcea26beaedc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:31.929814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-229ecfc6-f706-46b0-9012-dcea26beaedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:32.210921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ecfc6-f706-46b0-9012-dcea26beaedc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:32.264898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:32.330115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:32.330749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:32.331017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:32.331291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:32.332904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:32.431061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:32.432645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7396ab6-c858-411e-af12-2c2fc4a5882b service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:32.452042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 676/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:34.697345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603d85d-de37-444d-9330-003e23ec5fef None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:34.697863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603d85d-de37-444d-9330-003e23ec5fef None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:34.697943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603d85d-de37-444d-9330-003e23ec5fef None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:34.699426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 701/1377] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:34.705334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc3187-9c8e-4312-a854-2238f7010f05 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:34.705790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc3187-9c8e-4312-a854-2238f7010f05 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:34.706195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc3187-9c8e-4312-a854-2238f7010f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:34.786015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b3fc3187-9c8e-4312-a854-2238f7010f05 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:34.787403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b3fc3187-9c8e-4312-a854-2238f7010f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:34.795342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:34.898325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 677/1378] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:35.184338 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603d85d-de37-444d-9330-003e23ec5fef None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:35.185843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5603d85d-de37-444d-9330-003e23ec5fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:35.345760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603d85d-de37-444d-9330-003e23ec5fef None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:35.349962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:35.469411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:35.469411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:35.469701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:35.469983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:35.471516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:35.567554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:35.569159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a06ac47-8742-4580-883e-2b4ac546ef92 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:35.589324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 702/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:37.879633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef60c2-e39a-4eb8-b0d9-8788e7798f5e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:37.880053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef60c2-e39a-4eb8-b0d9-8788e7798f5e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:37.880258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ef60c2-e39a-4eb8-b0d9-8788e7798f5e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:37.881483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 678/1380] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:37.886870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c94e97-3996-4502-b91f-0814763a456a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:37.888300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c94e97-3996-4502-b91f-0814763a456a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:37.888445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c94e97-3996-4502-b91f-0814763a456a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:37.906080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:38.008284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-79c94e97-3996-4502-b91f-0814763a456a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:38.010852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-79c94e97-3996-4502-b91f-0814763a456a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:38.111187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 703/1381] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:38.402968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef60c2-e39a-4eb8-b0d9-8788e7798f5e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:38.404478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b2ef60c2-e39a-4eb8-b0d9-8788e7798f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:38.571080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:38.584426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ef60c2-e39a-4eb8-b0d9-8788e7798f5e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:38.705883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:38.706455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:38.706736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:38.707013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:38.708663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:38.804660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:38.806183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985f000c-b6b2-46e7-bbf8-9711eb0096e2 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:38.816703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:38.825487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 679/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:41.038826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484d51-6d7a-456e-85f0-af7f95774ede None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:41.039354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484d51-6d7a-456e-85f0-af7f95774ede None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:41.039354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a484d51-6d7a-456e-85f0-af7f95774ede None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:41.040706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 704/1383] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:41.046173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923481e1-5e86-4919-9119-d110562323f9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:41.047203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923481e1-5e86-4919-9119-d110562323f9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:41.047516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923481e1-5e86-4919-9119-d110562323f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:41.128329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-923481e1-5e86-4919-9119-d110562323f9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:41.129629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-923481e1-5e86-4919-9119-d110562323f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:41.247487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 680/1384] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:41.596428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484d51-6d7a-456e-85f0-af7f95774ede None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:41.598029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4a484d51-6d7a-456e-85f0-af7f95774ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:41.693098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:41.757556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a484d51-6d7a-456e-85f0-af7f95774ede None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:41.872982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:41.873589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:41.873861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:41.874138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:41.876388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:41.929217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:41.970429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:41.972034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098aa37e-2143-4b8f-87ef-ad3d161599fd service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:41.990072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 705/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:44.520895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a24a02-a2e7-43c7-80d2-5161474552e4 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:44.521474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a24a02-a2e7-43c7-80d2-5161474552e4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:44.521617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a24a02-a2e7-43c7-80d2-5161474552e4 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:44.523285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 681/1386] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:44.528754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62e4b0-b867-47de-a79b-1637fcbe9035 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:44.529161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62e4b0-b867-47de-a79b-1637fcbe9035 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:44.529325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe62e4b0-b867-47de-a79b-1637fcbe9035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:44.603546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fe62e4b0-b867-47de-a79b-1637fcbe9035 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:44.605180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fe62e4b0-b867-47de-a79b-1637fcbe9035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:44.708278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 706/1387] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:45.035477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a24a02-a2e7-43c7-80d2-5161474552e4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:45.037312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a2a24a02-a2e7-43c7-80d2-5161474552e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:45.121279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:45.197055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a24a02-a2e7-43c7-80d2-5161474552e4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:45.312718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:45.313244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:45.313514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:45.313780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:45.315374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:45.355197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:45.408441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:45.410494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256cb0ca-9481-4f38-9ea3-dd03b72ca7f5 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:45.431858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 682/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:47.612378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c70e42-3f12-4db5-a3b6-a34506d102a7 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:47.612916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c70e42-3f12-4db5-a3b6-a34506d102a7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:47.612916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c70e42-3f12-4db5-a3b6-a34506d102a7 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:47.614473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 707/1389] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:47.619448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9170-775e-4629-b358-a69621d706ca None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:47.619840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9170-775e-4629-b358-a69621d706ca None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:47.619995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bcf9170-775e-4629-b358-a69621d706ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:47.687097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5bcf9170-775e-4629-b358-a69621d706ca None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:47.688472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5bcf9170-775e-4629-b358-a69621d706ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:47.785117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 683/1390] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:48.075656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c70e42-3f12-4db5-a3b6-a34506d102a7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:48.077176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-11c70e42-3f12-4db5-a3b6-a34506d102a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:48.111090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:48.230089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c70e42-3f12-4db5-a3b6-a34506d102a7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:48.335753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:48.347696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:48.348300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.348575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.348840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.350486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:48.354272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2600f2-9d28-4a04-af71-373ac8f172fe None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.354559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2600f2-9d28-4a04-af71-373ac8f172fe None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.354836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2600f2-9d28-4a04-af71-373ac8f172fe None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.356175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 684/1391] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:13:48.361743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46fde9-7dc8-4352-8d3e-4b78f174d966 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.362018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46fde9-7dc8-4352-8d3e-4b78f174d966 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.362340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a46fde9-7dc8-4352-8d3e-4b78f174d966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.377839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:48.429658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8a46fde9-7dc8-4352-8d3e-4b78f174d966 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:48.430911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8a46fde9-7dc8-4352-8d3e-4b78f174d966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:48.444641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:48.446281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55264ece-cc2c-4d46-abab-757d2e4df755 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:48.464516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 708/1392] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:48.532813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 685/1393] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:13:48.835815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2385-d6af-4c56-82d9-719c1f73cc26 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.836259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2385-d6af-4c56-82d9-719c1f73cc26 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.836497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16e2385-d6af-4c56-82d9-719c1f73cc26 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.837868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 709/1394] 10.222.0.13 () {54 vars in 962 bytes} [Tue Oct 4 10:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:48.843230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a2e2-503f-460f-bf3c-abb6bfb2e51c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:48.843654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a2e2-503f-460f-bf3c-abb6bfb2e51c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:48.843881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d61a2e2-503f-460f-bf3c-abb6bfb2e51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:48.919700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1d61a2e2-503f-460f-bf3c-abb6bfb2e51c None None] MFA Rules not processed for user `d5c6c66598c74c11a1c9cac119644c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:48.920800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1d61a2e2-503f-460f-bf3c-abb6bfb2e51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:48.956142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:48.999517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 686/1395] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Oct 4 10:13:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:49.009192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e2385-d6af-4c56-82d9-719c1f73cc26 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:49.010792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b16e2385-d6af-4c56-82d9-719c1f73cc26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:49.116265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16e2385-d6af-4c56-82d9-719c1f73cc26 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:49.131677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:49.201219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:49.201742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:49.202016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:49.202330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:49.204306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:49.318813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:49.320260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c797b3-32c8-4b00-9c5a-ffd11c5b60af service glance] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:49.338638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 710/1396] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:50.104375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019a528-06c4-4cbf-95d9-a82b42c606ed None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:50.104880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019a528-06c4-4cbf-95d9-a82b42c606ed None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:50.104992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b019a528-06c4-4cbf-95d9-a82b42c606ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:50.176493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b019a528-06c4-4cbf-95d9-a82b42c606ed None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:50.177710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b019a528-06c4-4cbf-95d9-a82b42c606ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:50.262790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 687/1397] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:50.305642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e1b9f-6be8-47a4-bac2-496a2b7768bb None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:50.306157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e1b9f-6be8-47a4-bac2-496a2b7768bb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:50.306331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e1b9f-6be8-47a4-bac2-496a2b7768bb None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:50.307587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 711/1398] 10.222.0.13 () {54 vars in 943 bytes} [Tue Oct 4 10:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:50.314846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d99a1-7727-40fb-a1c4-69428817e3f6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:50.315313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d99a1-7727-40fb-a1c4-69428817e3f6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:50.315644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47d99a1-7727-40fb-a1c4-69428817e3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:50.373800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:50.393284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f47d99a1-7727-40fb-a1c4-69428817e3f6 None None] MFA Rules not processed for user `bf8cf97352f244d4b87c5927ee451dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:50.394509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f47d99a1-7727-40fb-a1c4-69428817e3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:50.504987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 688/1399] 10.222.0.13 () {58 vars in 1040 bytes} [Tue Oct 4 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:50.515575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e1b9f-6be8-47a4-bac2-496a2b7768bb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:50.517266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-271e1b9f-6be8-47a4-bac2-496a2b7768bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:50.567565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:50.667868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e1b9f-6be8-47a4-bac2-496a2b7768bb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:50.687153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7e183-d25b-47d0-9be6-793b18d73155 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:50.688266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7e183-d25b-47d0-9be6-793b18d73155 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:50.688547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc7e183-d25b-47d0-9be6-793b18d73155 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:50.689894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 689/1400] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:13:50.695116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:50.695393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:50.695663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:50.763584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:50.764857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:50.778824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:50.779433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:50.779707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:50.779975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:50.781635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:50.810902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:50.823101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:50.865946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 690/1401] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:13:50.867022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:50.868948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa52906-f107-4068-8083-eea677f21956 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:50.878412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 712/1402] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:13:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:13:51.159133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:51.160380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:51.293839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65074f7a-af36-4bee-acdc-6abbe9fe996b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:51.370016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:51.421390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:51.422041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:51.422352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:51.422621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:51.424207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:51.536308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:51.538049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7918797f-93ad-45dd-bae4-5b366967d043 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:51.557849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 691/1403] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:53.862728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb463167-625e-43d4-835f-64c9892019d4 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:53.863277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb463167-625e-43d4-835f-64c9892019d4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:53.863277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb463167-625e-43d4-835f-64c9892019d4 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:53.864641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 713/1404] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:53.869866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6a37-cc21-4ef4-aa24-2782d5555d7b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:53.870669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6a37-cc21-4ef4-aa24-2782d5555d7b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:53.871111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baf6a37-cc21-4ef4-aa24-2782d5555d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:53.942763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2baf6a37-cc21-4ef4-aa24-2782d5555d7b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:53.943895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2baf6a37-cc21-4ef4-aa24-2782d5555d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:53.979817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:54.054903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 692/1405] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:54.357042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb463167-625e-43d4-835f-64c9892019d4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:54.359851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cb463167-625e-43d4-835f-64c9892019d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:54.525033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb463167-625e-43d4-835f-64c9892019d4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:54.547029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:54.664412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:54.665114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:54.665456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:54.665789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:54.667494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:54.906698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:54.908550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989db6be-c525-4e8d-ac87-f1124b1b6695 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:54.934321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 714/1406] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:13:57.175537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e9fa2-5a0c-47f3-8b1e-e20ebc249c9e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.176413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e9fa2-5a0c-47f3-8b1e-e20ebc249c9e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.176695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531e9fa2-5a0c-47f3-8b1e-e20ebc249c9e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.178189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 693/1407] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:13:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:13:57.183537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db70c-addd-4b2e-ba40-a347b81a529e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.183962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db70c-addd-4b2e-ba40-a347b81a529e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.184052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736db70c-addd-4b2e-ba40-a347b81a529e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.226023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:57.261597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-736db70c-addd-4b2e-ba40-a347b81a529e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:13:57.262833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-736db70c-addd-4b2e-ba40-a347b81a529e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:57.367471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 715/1408] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:13:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:13:57.654704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e9fa2-5a0c-47f3-8b1e-e20ebc249c9e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:13:57.656901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-531e9fa2-5a0c-47f3-8b1e-e20ebc249c9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:57.824898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531e9fa2-5a0c-47f3-8b1e-e20ebc249c9e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:13:57.842519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:13:57.949241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:13:57.949883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:13:57.950250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:13:57.950595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:13:57.952390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:13:58.051843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:13:58.053719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9440fd-f1a0-4a00-b3f0-f4c2ec6d4f3f service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:13:58.078423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 694/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:13:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:07.365424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=53 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:08.071270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.166214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e503-877d-4273-b525-6091123612b4 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.166773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e503-877d-4273-b525-6091123612b4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.166773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7e503-877d-4273-b525-6091123612b4 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.168105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 716/1410] 10.222.0.13 () {54 vars in 918 bytes} [Tue Oct 4 10:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:14.174360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37f89-b310-481d-be27-b40b8744205f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.174777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37f89-b310-481d-be27-b40b8744205f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.174986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d37f89-b310-481d-be27-b40b8744205f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.255379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d8d37f89-b310-481d-be27-b40b8744205f None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:14.257116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d8d37f89-b310-481d-be27-b40b8744205f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:14.378480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 695/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Oct 4 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:14.393778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1f6e4-eae4-4821-944e-4d98e059cf84 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.394148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1f6e4-eae4-4821-944e-4d98e059cf84 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.394470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d1f6e4-eae4-4821-944e-4d98e059cf84 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.395842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 717/1412] 10.222.0.13 () {54 vars in 918 bytes} [Tue Oct 4 10:14:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:14.400704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71c1a6-30c6-4ee4-b605-083f58d78ce1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:14.400991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71c1a6-30c6-4ee4-b605-083f58d78ce1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:14.401257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71c1a6-30c6-4ee4-b605-083f58d78ce1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:14.485211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:14.489189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4f71c1a6-30c6-4ee4-b605-083f58d78ce1 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:14.490620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4f71c1a6-30c6-4ee4-b605-083f58d78ce1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:14.620699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 696/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Tue Oct 4 10:14:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:14:17.219393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d642-ffc0-428b-b0c0-b84b69e23449 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.220078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d642-ffc0-428b-b0c0-b84b69e23449 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.220078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c5d642-ffc0-428b-b0c0-b84b69e23449 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:17.221328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:17.226558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1b24-c8e6-4924-a35d-bc3887fc29ba None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.226954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1b24-c8e6-4924-a35d-bc3887fc29ba None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.227100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c1b24-c8e6-4924-a35d-bc3887fc29ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:17.291692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cb2c1b24-c8e6-4924-a35d-bc3887fc29ba None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:17.292882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cb2c1b24-c8e6-4924-a35d-bc3887fc29ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:17.347353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:17.388651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:17.676153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5d642-ffc0-428b-b0c0-b84b69e23449 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:17.677726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-90c5d642-ffc0-428b-b0c0-b84b69e23449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:17.822864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c5d642-ffc0-428b-b0c0-b84b69e23449 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:17.897888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:17.938138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:17.938842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:17.939111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:17.939371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:17.940944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:18.032621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:18.034238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c389a-69cf-435f-9254-33ae8fcd9f65 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:18.052579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 719/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:19.198181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de481a-247e-4750-80ad-d737d6db1e6c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.198728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de481a-247e-4750-80ad-d737d6db1e6c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.198728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de481a-247e-4750-80ad-d737d6db1e6c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.200091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:19.206512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827a336-62c9-4f0b-ab4f-b60f2b39d203 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.206938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827a336-62c9-4f0b-ab4f-b60f2b39d203 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.207072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5827a336-62c9-4f0b-ab4f-b60f2b39d203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.280580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5827a336-62c9-4f0b-ab4f-b60f2b39d203 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:19.282472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5827a336-62c9-4f0b-ab4f-b60f2b39d203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:19.308655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:19.376280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:19.523687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de481a-247e-4750-80ad-d737d6db1e6c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:19.525214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-55de481a-247e-4750-80ad-d737d6db1e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:19.669835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de481a-247e-4750-80ad-d737d6db1e6c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:19.716489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:19.788135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:19.788726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:19.788996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:19.789259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:19.790849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:19.877491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:19.879082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2acb59-b7eb-46bb-bb0c-4d6fea7cf3ec service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:19.896369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 699/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:21.048598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e78d0e-2f19-43cd-9357-9a61ba68fa26 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.049161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e78d0e-2f19-43cd-9357-9a61ba68fa26 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.049161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e78d0e-2f19-43cd-9357-9a61ba68fa26 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.050541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:21.056760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e91d6-55c8-4a4b-bde4-3e9def88a0b9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.057199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e91d6-55c8-4a4b-bde4-3e9def88a0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.057297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e91d6-55c8-4a4b-bde4-3e9def88a0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.112206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.133415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4d9e91d6-55c8-4a4b-bde4-3e9def88a0b9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:21.134639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4d9e91d6-55c8-4a4b-bde4-3e9def88a0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:21.228415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:21.377914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e78d0e-2f19-43cd-9357-9a61ba68fa26 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:21.379478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-28e78d0e-2f19-43cd-9357-9a61ba68fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:21.523766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e78d0e-2f19-43cd-9357-9a61ba68fa26 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:21.535407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:21.634571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:21.635201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:21.635475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:21.635776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:21.637494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:21.731554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:21.733136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b770a-ad63-4987-a1f6-c5bc8db4d2d2 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:21.751000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 722/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:22.879984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83711-3d20-4f80-ae2d-dfb3166b29e2 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:22.880510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83711-3d20-4f80-ae2d-dfb3166b29e2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:22.880510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc83711-3d20-4f80-ae2d-dfb3166b29e2 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:22.881855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:22.888122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed5832-2158-4862-92af-792a65bdf1ed None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:22.888596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed5832-2158-4862-92af-792a65bdf1ed None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:22.888695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed5832-2158-4862-92af-792a65bdf1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:22.908435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:22.963185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d9ed5832-2158-4862-92af-792a65bdf1ed None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:22.964373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d9ed5832-2158-4862-92af-792a65bdf1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:23.060396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:23.211031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83711-3d20-4f80-ae2d-dfb3166b29e2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:23.212752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6bc83711-3d20-4f80-ae2d-dfb3166b29e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:23.327377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:23.363562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc83711-3d20-4f80-ae2d-dfb3166b29e2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:23.480217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:23.480737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:23.481013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:23.481282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:23.482973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:23.557576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:23.572151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:23.573754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13bd21a-d3d8-4c3a-821c-ef21450612d9 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:23.593802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 702/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:24.720075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bfca8-6899-49c4-bccb-d935cc09be25 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.720621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bfca8-6899-49c4-bccb-d935cc09be25 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.720621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55bfca8-6899-49c4-bccb-d935cc09be25 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.721959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:24.728219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade611e-518e-437d-a913-dcd5d262ba66 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:24.728665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade611e-518e-437d-a913-dcd5d262ba66 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:24.728766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade611e-518e-437d-a913-dcd5d262ba66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:24.803307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cade611e-518e-437d-a913-dcd5d262ba66 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:24.804497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cade611e-518e-437d-a913-dcd5d262ba66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:24.899197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:25.048077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bfca8-6899-49c4-bccb-d935cc09be25 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:25.049599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a55bfca8-6899-49c4-bccb-d935cc09be25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:25.133848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.195357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55bfca8-6899-49c4-bccb-d935cc09be25 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:25.306725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:25.307322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:25.307592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:25.307854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:25.309431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:25.357148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:25.398022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:25.399650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d9e171-1d9a-4a4b-a06d-862564b23f94 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:25.417686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 725/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:26.467085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acc773-5086-4c95-b203-4a24618fab13 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:26.467653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acc773-5086-4c95-b203-4a24618fab13 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:26.467653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3acc773-5086-4c95-b203-4a24618fab13 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:26.469005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:26.475410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a44ff-16a0-43d0-b931-3703683f9402 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:26.475842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a44ff-16a0-43d0-b931-3703683f9402 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:26.475962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65a44ff-16a0-43d0-b931-3703683f9402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:26.550156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e65a44ff-16a0-43d0-b931-3703683f9402 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:26.551372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e65a44ff-16a0-43d0-b931-3703683f9402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:26.648375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:26.800634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acc773-5086-4c95-b203-4a24618fab13 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:26.802046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b3acc773-5086-4c95-b203-4a24618fab13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:26.854402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:26.949228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3acc773-5086-4c95-b203-4a24618fab13 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:27.069598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:27.070126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:27.070424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:27.070693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:27.072263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:27.079678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:27.160984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:27.162569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccae364-691e-4070-91c6-a253490aa6a3 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:27.179729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 705/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:28.225250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d3cd-7446-4bf0-87ab-ec587a3ac13f None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.225813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d3cd-7446-4bf0-87ab-ec587a3ac13f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.225813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4956d3cd-7446-4bf0-87ab-ec587a3ac13f None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.227212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:28.233236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995d21-d176-416d-8190-f4f3b3082997 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.233678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995d21-d176-416d-8190-f4f3b3082997 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.233791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71995d21-d176-416d-8190-f4f3b3082997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.307972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-71995d21-d176-416d-8190-f4f3b3082997 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:28.309157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-71995d21-d176-416d-8190-f4f3b3082997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.376256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.403464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:28.554195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d3cd-7446-4bf0-87ab-ec587a3ac13f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:28.555666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4956d3cd-7446-4bf0-87ab-ec587a3ac13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.575693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.701727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4956d3cd-7446-4bf0-87ab-ec587a3ac13f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:28.788506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:28.814442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:28.814974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:28.815244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:28.815503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:28.817081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:28.903898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:28.905467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45947b4-1cbb-43cf-8252-6b7ec9b299f5 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:28.924035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 728/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:30.005530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955ddbf-3e83-4ea4-b692-bd21659a2fb7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.006066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955ddbf-3e83-4ea4-b692-bd21659a2fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.006066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b955ddbf-3e83-4ea4-b692-bd21659a2fb7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.007598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:30.014165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399d877-1833-4d94-8948-a4cac817bece None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.014625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399d877-1833-4d94-8948-a4cac817bece None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.014741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4399d877-1833-4d94-8948-a4cac817bece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.091755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4399d877-1833-4d94-8948-a4cac817bece None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:30.092965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4399d877-1833-4d94-8948-a4cac817bece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:30.134402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:30.188284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:30.337434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955ddbf-3e83-4ea4-b692-bd21659a2fb7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:30.338900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b955ddbf-3e83-4ea4-b692-bd21659a2fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:30.482807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b955ddbf-3e83-4ea4-b692-bd21659a2fb7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:30.536616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:30.593735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:30.594321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:30.594627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:30.594896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:30.596486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:30.683967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:30.685536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a44c0-0d28-4928-ba0d-dfefbdf20e30 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:30.704049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 708/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:31.749125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79305b0-4aa0-4deb-8d02-e481c7aabee5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:31.749623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79305b0-4aa0-4deb-8d02-e481c7aabee5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:31.749623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79305b0-4aa0-4deb-8d02-e481c7aabee5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:31.751026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:31.757190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bcb9-1518-4e83-a930-ecda14cff800 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:31.757638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bcb9-1518-4e83-a930-ecda14cff800 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:31.757738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3887bcb9-1518-4e83-a930-ecda14cff800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:31.834289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3887bcb9-1518-4e83-a930-ecda14cff800 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:31.835474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3887bcb9-1518-4e83-a930-ecda14cff800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:31.837684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:31.928534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:14:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:32.077021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79305b0-4aa0-4deb-8d02-e481c7aabee5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:32.079147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d79305b0-4aa0-4deb-8d02-e481c7aabee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:32.220073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79305b0-4aa0-4deb-8d02-e481c7aabee5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:32.245810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:32.329778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:32.330320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:32.330595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:32.330858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:32.332437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:32.419131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:32.420727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefa45e1-11aa-4997-9f85-3131f45e7065 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:32.439620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 731/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:34.531438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d026d-4f81-437b-9a1a-96c3dccb09d5 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.531973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d026d-4f81-437b-9a1a-96c3dccb09d5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.532098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d026d-4f81-437b-9a1a-96c3dccb09d5 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.533436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:34.538776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8ca41-6283-40a8-a82a-aa4e0bf2c30d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:34.539204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8ca41-6283-40a8-a82a-aa4e0bf2c30d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:34.539342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb8ca41-6283-40a8-a82a-aa4e0bf2c30d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:34.580808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:34.606168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7eb8ca41-6283-40a8-a82a-aa4e0bf2c30d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:34.607416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7eb8ca41-6283-40a8-a82a-aa4e0bf2c30d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:34.710047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:34.997708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d026d-4f81-437b-9a1a-96c3dccb09d5 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:34.999296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6a7d026d-4f81-437b-9a1a-96c3dccb09d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:35.101896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d026d-4f81-437b-9a1a-96c3dccb09d5 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:35.134986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:35.177286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:35.177886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:35.178162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:35.178470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:35.180052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:35.291602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:35.292983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48f1988-57c0-405f-939c-56697a7bd2e6 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:35.315524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 711/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:37.607480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5073b-e284-495c-962b-d58b2aa795b9 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.608020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5073b-e284-495c-962b-d58b2aa795b9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.608020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c5073b-e284-495c-962b-d58b2aa795b9 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.609377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:37.615871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd275a8-a234-4ac3-bfbe-716d1331fc0c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:37.616317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd275a8-a234-4ac3-bfbe-716d1331fc0c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:37.616409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd275a8-a234-4ac3-bfbe-716d1331fc0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:37.682447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5cd275a8-a234-4ac3-bfbe-716d1331fc0c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:37.683630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5cd275a8-a234-4ac3-bfbe-716d1331fc0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:37.696574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:37.778562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:38.062072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5073b-e284-495c-962b-d58b2aa795b9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:38.063736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d4c5073b-e284-495c-962b-d58b2aa795b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:38.166137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c5073b-e284-495c-962b-d58b2aa795b9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:38.179604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:38.243354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:38.243951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:38.244227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:38.244494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:38.246081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:38.355138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:38.356572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc3c7-9027-46c4-b56c-41bd4d1d7336 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:38.374172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 734/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:40.928022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cd6b0-91c6-426a-b8ee-cb46cec71504 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.928528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cd6b0-91c6-426a-b8ee-cb46cec71504 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.928528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339cd6b0-91c6-426a-b8ee-cb46cec71504 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.929819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:40.935065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de20e3-ae73-4b33-a062-94d43dfc16c9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:40.936184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de20e3-ae73-4b33-a062-94d43dfc16c9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:40.936465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de20e3-ae73-4b33-a062-94d43dfc16c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:40.983352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.002368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c2de20e3-ae73-4b33-a062-94d43dfc16c9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:41.003569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c2de20e3-ae73-4b33-a062-94d43dfc16c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:41.098952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:41.400328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cd6b0-91c6-426a-b8ee-cb46cec71504 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:41.401847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-339cd6b0-91c6-426a-b8ee-cb46cec71504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:41.504441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339cd6b0-91c6-426a-b8ee-cb46cec71504 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:41.543649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:41.577164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:41.577762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:41.578033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:41.578879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:41.580507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:41.686384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:41.688274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69115392-e7e0-4b63-9866-e414b3167a2c service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:41.706000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 714/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:43.985026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27b8f6-2563-4922-a8db-3dcb6c30fe7a None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:43.985549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27b8f6-2563-4922-a8db-3dcb6c30fe7a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:43.985647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e27b8f6-2563-4922-a8db-3dcb6c30fe7a None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:43.987065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:43.992624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee094f56-a7b2-4243-b164-0fe3b0944cb7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:43.993008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee094f56-a7b2-4243-b164-0fe3b0944cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:43.993160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee094f56-a7b2-4243-b164-0fe3b0944cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.059764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ee094f56-a7b2-4243-b164-0fe3b0944cb7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:44.060961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ee094f56-a7b2-4243-b164-0fe3b0944cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:44.082021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.159454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:44.444779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27b8f6-2563-4922-a8db-3dcb6c30fe7a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:44.446470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3e27b8f6-2563-4922-a8db-3dcb6c30fe7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:44.547319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e27b8f6-2563-4922-a8db-3dcb6c30fe7a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:44.566618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:44.619536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:44.620137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:44.620408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:44.620671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:44.622327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:44.742796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:44.744150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060d0f41-825a-44ac-94ca-b8e22016ef8d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:44.768020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 737/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:46.942857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcb76d-7ca5-4be5-af9d-9ce5ad3491e0 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:46.943363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcb76d-7ca5-4be5-af9d-9ce5ad3491e0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:46.943363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbcb76d-7ca5-4be5-af9d-9ce5ad3491e0 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:46.944644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:46.949861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea80893-3a46-4a26-8080-678d18914b12 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:46.950331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea80893-3a46-4a26-8080-678d18914b12 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:46.950437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea80893-3a46-4a26-8080-678d18914b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:47.016510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:47.019519 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1ea80893-3a46-4a26-8080-678d18914b12 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:47.021389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1ea80893-3a46-4a26-8080-678d18914b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:47.116527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:14:47.402911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcb76d-7ca5-4be5-af9d-9ce5ad3491e0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:47.404496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1bbcb76d-7ca5-4be5-af9d-9ce5ad3491e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:47.506622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbcb76d-7ca5-4be5-af9d-9ce5ad3491e0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:47.555514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:47.579898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:47.580504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:47.580775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:47.581040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:47.582629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:47.690174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:47.691576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2cc3b0-571e-46a6-9981-86ffaf32f583 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:47.709577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 717/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:49.925077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aebc5-f918-4802-afd3-939fe784fa43 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.925611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aebc5-f918-4802-afd3-939fe784fa43 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.925611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574aebc5-f918-4802-afd3-939fe784fa43 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.926985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:49.932334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76b12-b368-47bc-8984-243530414ed8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:49.932752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76b12-b368-47bc-8984-243530414ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:49.932876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c76b12-b368-47bc-8984-243530414ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:49.997323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a0c76b12-b368-47bc-8984-243530414ed8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:49.998570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a0c76b12-b368-47bc-8984-243530414ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:50.028157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.097619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:50.381399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aebc5-f918-4802-afd3-939fe784fa43 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:50.383009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-574aebc5-f918-4802-afd3-939fe784fa43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:50.485423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574aebc5-f918-4802-afd3-939fe784fa43 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:50.518875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:50.564355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:50.564959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:50.565237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:50.565509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:50.567180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:50.679012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:50.680495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6f76f-5ea9-46f6-b2ad-72209c03c8c9 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:50.699016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 740/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:52.927389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115893-c56d-4f0e-b770-da01ec2559c2 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.927947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115893-c56d-4f0e-b770-da01ec2559c2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.927947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4115893-c56d-4f0e-b770-da01ec2559c2 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:52.929208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:52.934399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83b703-5285-4387-8ff3-c0b0c232fa9a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:52.934826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83b703-5285-4387-8ff3-c0b0c232fa9a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:52.934927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83b703-5285-4387-8ff3-c0b0c232fa9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.005190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9f83b703-5285-4387-8ff3-c0b0c232fa9a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:53.006454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9f83b703-5285-4387-8ff3-c0b0c232fa9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:53.015082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:53.106085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:14:53.393998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4115893-c56d-4f0e-b770-da01ec2559c2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:53.395545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e4115893-c56d-4f0e-b770-da01ec2559c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:53.497610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4115893-c56d-4f0e-b770-da01ec2559c2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:53.552152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:53.569716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:53.570348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:53.570628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:53.570900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:53.572475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:53.680464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:53.681880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd04e143-4c38-486f-be91-8d3137754be1 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:53.700034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 720/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:55.885606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de5375-b727-42d5-aa06-fa566e565c67 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.886132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de5375-b727-42d5-aa06-fa566e565c67 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.886132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29de5375-b727-42d5-aa06-fa566e565c67 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.887488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:55.893011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04948d3-06f6-49fd-9f38-95c027ed5099 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:55.893436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04948d3-06f6-49fd-9f38-95c027ed5099 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:55.893554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04948d3-06f6-49fd-9f38-95c027ed5099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:55.957877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f04948d3-06f6-49fd-9f38-95c027ed5099 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:55.959088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f04948d3-06f6-49fd-9f38-95c027ed5099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:55.996759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.053959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:14:56.339629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de5375-b727-42d5-aa06-fa566e565c67 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:56.341192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-29de5375-b727-42d5-aa06-fa566e565c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:56.443157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29de5375-b727-42d5-aa06-fa566e565c67 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:56.479142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:56.514530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:56.515123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:56.515398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:56.515699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:56.517480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:56.624478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:56.626872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4255d31c-3a7e-47e0-ae27-93083663116b service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:56.644956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 743/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:14:58.796809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec758e6-23e9-4a59-a086-d31e30e43bbd None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.797411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec758e6-23e9-4a59-a086-d31e30e43bbd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.797411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec758e6-23e9-4a59-a086-d31e30e43bbd None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.798857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:14:58.807059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdd2de-16eb-436c-b57f-a1ef830cc4b0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:58.807504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdd2de-16eb-436c-b57f-a1ef830cc4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:58.807618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cdd2de-16eb-436c-b57f-a1ef830cc4b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:58.873964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-56cdd2de-16eb-436c-b57f-a1ef830cc4b0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:14:58.875757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-56cdd2de-16eb-436c-b57f-a1ef830cc4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:58.893718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:58.969430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:14:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:14:59.261020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec758e6-23e9-4a59-a086-d31e30e43bbd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:14:59.262594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dec758e6-23e9-4a59-a086-d31e30e43bbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:59.365546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec758e6-23e9-4a59-a086-d31e30e43bbd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:14:59.430734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:14:59.437425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:14:59.438615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:14:59.438886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:14:59.439154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:14:59.440737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:14:59.545925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:14:59.548063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca2738-ecc8-4a75-8888-69d0b9140dd1 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:14:59.566403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 723/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:14:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:01.741304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35710c26-acb5-4238-9d41-682013873243 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.742027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35710c26-acb5-4238-9d41-682013873243 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.742027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35710c26-acb5-4238-9d41-682013873243 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.743234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:01.748933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959805e8-80f7-45a8-b612-60225c57c016 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:01.749321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959805e8-80f7-45a8-b612-60225c57c016 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:01.749489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959805e8-80f7-45a8-b612-60225c57c016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:01.815287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-959805e8-80f7-45a8-b612-60225c57c016 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:01.816487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-959805e8-80f7-45a8-b612-60225c57c016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:01.863946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:01.912077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:15:02.196002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35710c26-acb5-4238-9d41-682013873243 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:02.197533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-35710c26-acb5-4238-9d41-682013873243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:02.300374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35710c26-acb5-4238-9d41-682013873243 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:02.348200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:02.373813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:02.374515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:02.374792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:02.375641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:02.377243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:02.481523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:02.482918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431aff70-9cc6-4e6d-b279-52bc298dfad6 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:02.501390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 746/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:11.910485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=22 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:12.498507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=66 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:22.691527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb17130-b7d8-4a3e-a2b7-bacc0eb14297 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.692118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb17130-b7d8-4a3e-a2b7-bacc0eb14297 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.692118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb17130-b7d8-4a3e-a2b7-bacc0eb14297 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.693404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 725/1471] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:22.699972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4745b-28bd-4613-a53d-d7f8d1ac265c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:22.700457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4745b-28bd-4613-a53d-d7f8d1ac265c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:22.700562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e4745b-28bd-4613-a53d-d7f8d1ac265c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:22.771349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-90e4745b-28bd-4613-a53d-d7f8d1ac265c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:22.772561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-90e4745b-28bd-4613-a53d-d7f8d1ac265c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:22.882194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 747/1472] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:15:23.183995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859344ae-c08e-41d7-b45e-536c4bc221eb None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.184276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859344ae-c08e-41d7-b45e-536c4bc221eb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.184551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859344ae-c08e-41d7-b45e-536c4bc221eb None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.185909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 726/1473] 10.222.0.13 () {54 vars in 962 bytes} [Tue Oct 4 10:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:15:23.192268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8226c6-b038-4dc5-8891-c00070197ccd None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.192558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8226c6-b038-4dc5-8891-c00070197ccd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.192841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8226c6-b038-4dc5-8891-c00070197ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.283592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9f8226c6-b038-4dc5-8891-c00070197ccd None None] MFA Rules not processed for user `d5c6c66598c74c11a1c9cac119644c19`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:23.284811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9f8226c6-b038-4dc5-8891-c00070197ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:23.308809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:23.369601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 748/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Tue Oct 4 10:15:23 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:15:23.378229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859344ae-c08e-41d7-b45e-536c4bc221eb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:23.379743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-859344ae-c08e-41d7-b45e-536c4bc221eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:23.486869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859344ae-c08e-41d7-b45e-536c4bc221eb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:23.581929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c856935-2283-472b-b916-86f44daa691e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:23.582568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c856935-2283-472b-b916-86f44daa691e service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:23.582836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c856935-2283-472b-b916-86f44daa691e service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:23.583104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c856935-2283-472b-b916-86f44daa691e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:23.584677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2c856935-2283-472b-b916-86f44daa691e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:23.651566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:23.716142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c856935-2283-472b-b916-86f44daa691e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:23.717611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c856935-2283-472b-b916-86f44daa691e service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:23.737488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 727/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:25.983720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e570cf7-c9bb-4078-ba52-fbe150eef4f5 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.984292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e570cf7-c9bb-4078-ba52-fbe150eef4f5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.984292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e570cf7-c9bb-4078-ba52-fbe150eef4f5 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:25.986137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 749/1476] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:25.991498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb9ff5-2f10-484f-ba50-4385fc44e691 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:25.991933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb9ff5-2f10-484f-ba50-4385fc44e691 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:25.992054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efb9ff5-2f10-484f-ba50-4385fc44e691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.056323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8efb9ff5-2f10-484f-ba50-4385fc44e691 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:26.057549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8efb9ff5-2f10-484f-ba50-4385fc44e691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:26.153325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 728/1477] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:15:26.423913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e570cf7-c9bb-4078-ba52-fbe150eef4f5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:26.425460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6e570cf7-c9bb-4078-ba52-fbe150eef4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:26.547771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e570cf7-c9bb-4078-ba52-fbe150eef4f5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:26.597357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:26.643651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:26.644256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:26.644528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:26.644794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:26.645513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:26.648358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:15:26.648358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:15:26.648358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:15:26.650485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960e51a-946a-43f2-9936-cf8feb4e16b3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:26.670818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 750/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Tue Oct 4 10:15:26 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:15:28.855236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5675-9f91-4f76-a4aa-b42ca9893f32 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.855798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5675-9f91-4f76-a4aa-b42ca9893f32 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.855798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2a5675-9f91-4f76-a4aa-b42ca9893f32 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.857139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 729/1479] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:28.863058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395b6b8-cae5-4e8e-a8b9-311d9dad065b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:28.863451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395b6b8-cae5-4e8e-a8b9-311d9dad065b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:28.863620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8395b6b8-cae5-4e8e-a8b9-311d9dad065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:28.928836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8395b6b8-cae5-4e8e-a8b9-311d9dad065b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:28.929950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8395b6b8-cae5-4e8e-a8b9-311d9dad065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:29.024966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 751/1480] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:15:29.334878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5675-9f91-4f76-a4aa-b42ca9893f32 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:29.336498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7a2a5675-9f91-4f76-a4aa-b42ca9893f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:29.347231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.487532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2a5675-9f91-4f76-a4aa-b42ca9893f32 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:29.567453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:29.609005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:29.610068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:29.610605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:29.611035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:29.612656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:29.709213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:29.710836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce591c-7316-449b-afab-cd037252e66c service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:29.729074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 730/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:15:32.062187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3bef4-1af5-4a5c-b5c8-450f61964d36 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.063360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3bef4-1af5-4a5c-b5c8-450f61964d36 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.063360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f3bef4-1af5-4a5c-b5c8-450f61964d36 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.064607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 752/1482] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:32.069825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6508ab-2613-4011-82ab-6b66c77a3991 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.070301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6508ab-2613-4011-82ab-6b66c77a3991 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.070413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6508ab-2613-4011-82ab-6b66c77a3991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.134796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-fb6508ab-2613-4011-82ab-6b66c77a3991 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:32.135973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fb6508ab-2613-4011-82ab-6b66c77a3991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:32.183655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.233803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 731/1483] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:15:32.523776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3bef4-1af5-4a5c-b5c8-450f61964d36 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:32.525322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-52f3bef4-1af5-4a5c-b5c8-450f61964d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:32.549810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.632342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f3bef4-1af5-4a5c-b5c8-450f61964d36 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:32.704189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:32.704887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:32.705158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:32.705423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:32.707060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:32.780741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:32.819327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:32.820755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fb42d-3afa-4aec-8352-3c21b2c94c84 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:32.839310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 753/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:35.064652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d90f6-b2c1-44c2-b16e-f905ae6a4ad3 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:35.065196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d90f6-b2c1-44c2-b16e-f905ae6a4ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:35.065196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d90f6-b2c1-44c2-b16e-f905ae6a4ad3 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:35.066584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 732/1485] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:35.071923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772c24-0e7a-419f-8c8f-0d45cfe34d11 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:35.072330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772c24-0e7a-419f-8c8f-0d45cfe34d11 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:35.072498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f772c24-0e7a-419f-8c8f-0d45cfe34d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:35.138582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9f772c24-0e7a-419f-8c8f-0d45cfe34d11 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:35.139787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9f772c24-0e7a-419f-8c8f-0d45cfe34d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:35.239065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 754/1486] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:15:35.542625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d90f6-b2c1-44c2-b16e-f905ae6a4ad3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:35.544206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d13d90f6-b2c1-44c2-b16e-f905ae6a4ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:35.674527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d90f6-b2c1-44c2-b16e-f905ae6a4ad3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:35.745300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:35.746966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:35.747559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:35.747840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:35.748306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:35.750715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:35.858884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:35.861011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ceda32-ab88-4374-b69b-ebcb47247060 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:35.879062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 733/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:38.210565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b665586-814a-48b5-9c88-1b851e1eb7d0 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.211133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b665586-814a-48b5-9c88-1b851e1eb7d0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.211133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b665586-814a-48b5-9c88-1b851e1eb7d0 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.212466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 755/1488] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:38.217995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9168f-411f-45ab-a733-52e5b61d6d86 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.218484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9168f-411f-45ab-a733-52e5b61d6d86 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.219072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c9168f-411f-45ab-a733-52e5b61d6d86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.285769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-61c9168f-411f-45ab-a733-52e5b61d6d86 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:38.287007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-61c9168f-411f-45ab-a733-52e5b61d6d86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:38.338671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.382093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 734/1489] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:15:38.699871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b665586-814a-48b5-9c88-1b851e1eb7d0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:38.701490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5b665586-814a-48b5-9c88-1b851e1eb7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:38.738016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:38.806392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b665586-814a-48b5-9c88-1b851e1eb7d0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:38.883265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:38.883790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:38.884064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:38.884333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:38.885932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:38.978923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:39.016461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:39.017903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72758f0-8a6b-42df-a0d2-b5ec6cba61e3 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:39.035732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 756/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:41.497689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494ff42-993c-49b4-b826-591b7aaaec99 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.498287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494ff42-993c-49b4-b826-591b7aaaec99 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.498287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9494ff42-993c-49b4-b826-591b7aaaec99 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.499604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 735/1491] 10.222.0.13 () {54 vars in 927 bytes} [Tue Oct 4 10:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:15:41.505738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89438802-fb3e-4237-a033-109be907d6d1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:41.506229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89438802-fb3e-4237-a033-109be907d6d1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:41.506359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89438802-fb3e-4237-a033-109be907d6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:41.571917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-89438802-fb3e-4237-a033-109be907d6d1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:15:41.573164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-89438802-fb3e-4237-a033-109be907d6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:41.670701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 757/1492] 10.222.0.13 () {58 vars in 1024 bytes} [Tue Oct 4 10:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:15:41.965536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494ff42-993c-49b4-b826-591b7aaaec99 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:15:41.967111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9494ff42-993c-49b4-b826-591b7aaaec99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:42.068783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9494ff42-993c-49b4-b826-591b7aaaec99 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:15:42.141819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:15:42.142440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:15:42.142712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:15:42.142978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:15:42.144532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:15:42.152768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:42.250630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:15:42.251963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eec1ad7-49a0-48a6-89ae-d8c234053c6d service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:15:42.269595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 736/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:15:51.669133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=82 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:15:52.267340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=50 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:00.642458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:00.642954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:00.643058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:00.721643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:00.722890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:00.817426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 758/1494] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:17:00.825545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:00.826033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:00.826135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:00.900496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:00.901609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:00.994660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 737/1495] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:17:00.999410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.000603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:01.011764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b616942-3626-4b95-a68b-bd2a642e26d5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.073387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:01.107230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.107824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.108124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.108387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.109017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:17:01.109313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.110355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:01.110355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:17:01.110355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:17:01.111230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.122192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 759/1496] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:17:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:17:01.131114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.140823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a11349-886c-4509-9d4e-bc1ee9cc24e3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.201019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:01.235665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.236261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.236539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.236809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.237424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.239149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:01.239149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:17:01.239149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:17:01.240329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.295317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 738/1497] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:17:01 2022] POST /identity/v3/projects => generated 377 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:17:01.299490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.301139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:01.334514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e144ab56-cfce-4b1b-ab24-1609010870a9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.425970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.426598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.426876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.427147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.427781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.428796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:01.428796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:17:01.428796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:17:01.430000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.473774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 760/1498] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:17:01 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:17:01.477866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.479639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:01.542495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb46f49c-8fda-4522-a379-4421a584407b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.624884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:01.636209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.636814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.637090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.637360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.638001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.639041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:01.639041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:17:01.639041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:17:01.640203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.652479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 739/1499] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:17:01.656704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.658247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:01.669477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d4c8dd-b16a-43f2-8c02-85f318e66201 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.699379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:01.785539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.786047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.786351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.786620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.787250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.788267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:01.788267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:17:01.788267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:17:01.789136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1504f1c1-da1e-4fbc-aa5c-528b6def4335 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.801714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 761/1500] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:17:01.805828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:01.807382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:01.846517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993884-a899-49d8-8d33-b70366899969 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:01.941228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:01.941883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:01.942162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:01.942472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] PATH_INFO: `/v3/projects/5525a232b6104c5cac002f3510c5db2e/users/3b4a76fdd1744df599a09e1adf462e1c/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:01.964901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5525a232b6104c5cac002f3510c5db2e, user_id=3b4a76fdd1744df599a09e1adf462e1c, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:01.965921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:01.965921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:17:01.965921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:17:01.969211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426853b-9dbe-4ca4-8271-a9c00d2725e3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:01.993470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 740/1501] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:17:01 2022] PUT /identity/v3/projects/5525a232b6104c5cac002f3510c5db2e/users/3b4a76fdd1744df599a09e1adf462e1c/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:17:02.005944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6104e-c36d-468f-b144-6372a6546c1e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:02.006270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6104e-c36d-468f-b144-6372a6546c1e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:02.006575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff6104e-c36d-468f-b144-6372a6546c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:02.077435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8ff6104e-c36d-468f-b144-6372a6546c1e None None] MFA Rules not processed for user `3b4a76fdd1744df599a09e1adf462e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:02.080836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8ff6104e-c36d-468f-b144-6372a6546c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:02.159802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 762/1502] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:17:02.173788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b061a-c5b9-4db2-9599-720c024ce1d7 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:02.174068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b061a-c5b9-4db2-9599-720c024ce1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:02.174370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b061a-c5b9-4db2-9599-720c024ce1d7 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:02.175705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 741/1503] 10.222.0.13 () {54 vars in 963 bytes} [Tue Oct 4 10:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:02.180521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cb514-5ac3-4c5b-bdc4-0937f0902151 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:02.180804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cb514-5ac3-4c5b-bdc4-0937f0902151 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:02.181077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9cb514-5ac3-4c5b-bdc4-0937f0902151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:02.211613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:02.257052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fd9cb514-5ac3-4c5b-bdc4-0937f0902151 None None] MFA Rules not processed for user `99de0be14e684b3696a4570987119ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:02.258267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd9cb514-5ac3-4c5b-bdc4-0937f0902151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:02.331858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 763/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Tue Oct 4 10:17:02 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:17:02.339393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b061a-c5b9-4db2-9599-720c024ce1d7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:02.340981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ec5b061a-c5b9-4db2-9599-720c024ce1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:02.393948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:02.448517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b061a-c5b9-4db2-9599-720c024ce1d7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:02.521633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:02.522258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:02.522542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:02.522809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:02.524369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:02.620564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:02.621991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:02.626071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:02.640575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 742/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:17:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:17:02.686820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09f89-2e8e-4834-a387-fee926cecb9b None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:02.687253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09f89-2e8e-4834-a387-fee926cecb9b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:02.687349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e09f89-2e8e-4834-a387-fee926cecb9b None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:02.688725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 764/1506] 10.222.0.13 () {52 vars in 830 bytes} [Tue Oct 4 10:17:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:17:02.692530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:02.693767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:02.790836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5884dae8-54a5-4395-8453-49f1721f52fc service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:02.885453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:02.886090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:02.886360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:02.886633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] PATH_INFO: `/v3/users/3b4a76fdd1744df599a09e1adf462e1c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:02.899299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b4a76fdd1744df599a09e1adf462e1c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:02.900311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:02.900311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:17:02.900311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:17:02.901518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:02.911664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:02.968611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b4a76fdd1744df599a09e1adf462e1c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:02.969139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b4a76fdd1744df599a09e1adf462e1c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:02.982489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b4a76fdd1744df599a09e1adf462e1c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:02.983504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b4a76fdd1744df599a09e1adf462e1c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:03.007158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b4a76fdd1744df599a09e1adf462e1c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:03.021607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 743/1507] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:17:02 2022] DELETE /identity/v3/users/3b4a76fdd1744df599a09e1adf462e1c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:17:03.026020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:03.026356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:03.026629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:03.083533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:17:03.084734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:03.159674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:03.172367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 765/1508] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:17:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:17:03.181957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:03.183219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:03.281163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5332e8fd-e586-43a7-a759-101123c8ed4e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:03.335228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:03.357599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:03.358279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:03.358561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:03.358832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:03.360433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:03.476533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:03.477882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708439d-d9b6-4cf9-8bff-5a0372f28cd0 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:03.496408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 744/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:17:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:17:03.863935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:17:03.865228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:17:03.877751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7b5652-c4c2-4379-b62a-c7b4d9da1643 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:17:03.979690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:17:03.980314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:17:03.980589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:17:03.980858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] PATH_INFO: `/v3/projects/5525a232b6104c5cac002f3510c5db2e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:17:03.991883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=5525a232b6104c5cac002f3510c5db2e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:17:03.992844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:17:03.992844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:17:03.992844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:17:03.994024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:17:04.051518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5525a232b6104c5cac002f3510c5db2e', 'request_id': 'req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:04.064952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5525a232b6104c5cac002f3510c5db2e', 'request_id': 'req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:04.072082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5525a232b6104c5cac002f3510c5db2e', 'request_id': 'req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:04.098000 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] The token cache is being invalidate because project 5525a232b6104c5cac002f3510c5db2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:17:04.098321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:04.098753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2c05aa6c-408d-4b76-ba0b-eec96434b9ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:17:04.100162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 766/1510] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:17:03 2022] DELETE /identity/v3/projects/5525a232b6104c5cac002f3510c5db2e => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:17:13.494086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=62 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:17:14.095937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:34.246610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5ab78-7fc5-4c93-b53b-fd6420c4d00c None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:34.247197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5ab78-7fc5-4c93-b53b-fd6420c4d00c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:34.247197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c5ab78-7fc5-4c93-b53b-fd6420c4d00c None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:34.248497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:34.254110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf652c-6277-4aa6-948d-e6630492a85a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:34.254559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf652c-6277-4aa6-948d-e6630492a85a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:34.254712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf652c-6277-4aa6-948d-e6630492a85a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:34.339673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cdaf652c-6277-4aa6-948d-e6630492a85a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:34.340881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cdaf652c-6277-4aa6-948d-e6630492a85a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:34.442068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:18:34.597779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5ab78-7fc5-4c93-b53b-fd6420c4d00c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:34.599928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b8c5ab78-7fc5-4c93-b53b-fd6420c4d00c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:34.722784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c5ab78-7fc5-4c93-b53b-fd6420c4d00c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:34.728302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699afb1f-4657-4097-82b1-01064c7da927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:34.728897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699afb1f-4657-4097-82b1-01064c7da927 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:34.729204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699afb1f-4657-4097-82b1-01064c7da927 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:34.729477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699afb1f-4657-4097-82b1-01064c7da927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:34.737999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699afb1f-4657-4097-82b1-01064c7da927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:34.739601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699afb1f-4657-4097-82b1-01064c7da927 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:34.748718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 746/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:18:35.836132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0529c1-5bb6-4724-8a2b-8962e69cc2ba None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:35.836687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0529c1-5bb6-4724-8a2b-8962e69cc2ba None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:35.836687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0529c1-5bb6-4724-8a2b-8962e69cc2ba None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:35.838078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:18:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:35.844422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220c002-d214-4485-8433-84651132469b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:35.844879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220c002-d214-4485-8433-84651132469b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:35.845092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220c002-d214-4485-8433-84651132469b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:35.866328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1220c002-d214-4485-8433-84651132469b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:35.867547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1220c002-d214-4485-8433-84651132469b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:35.875416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 747/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:18:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:18:36.026631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0529c1-5bb6-4724-8a2b-8962e69cc2ba None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:36.041088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0529c1-5bb6-4724-8a2b-8962e69cc2ba None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:36.046434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd93a98d-9401-43a8-8991-b3a2b5acbd79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:36.047016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93a98d-9401-43a8-8991-b3a2b5acbd79 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:36.047290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93a98d-9401-43a8-8991-b3a2b5acbd79 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:36.047558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd93a98d-9401-43a8-8991-b3a2b5acbd79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:36.055402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93a98d-9401-43a8-8991-b3a2b5acbd79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:36.056967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd93a98d-9401-43a8-8991-b3a2b5acbd79 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:36.066604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 769/1516] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:18:36.224636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720387a6-8ecc-4e2d-904b-78aa5de2a29a None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:36.224835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720387a6-8ecc-4e2d-904b-78aa5de2a29a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:36.225107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720387a6-8ecc-4e2d-904b-78aa5de2a29a None None] PATH_INFO: `/v3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:36.226431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 748/1517] 10.222.0.36 () {54 vars in 925 bytes} [Tue Oct 4 10:18:36 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:18:36.232835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586acbc1-0cba-4a9f-aae3-5e34c8dcbab4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:36.233119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586acbc1-0cba-4a9f-aae3-5e34c8dcbab4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:36.233424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586acbc1-0cba-4a9f-aae3-5e34c8dcbab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:36.294576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-586acbc1-0cba-4a9f-aae3-5e34c8dcbab4 None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:36.295775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-586acbc1-0cba-4a9f-aae3-5e34c8dcbab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:36.312232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:18:36.367108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 770/1518] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Oct 4 10:18:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:18:45.093367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff25acf-f998-4b2d-b206-79efe7fef892 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:45.093806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff25acf-f998-4b2d-b206-79efe7fef892 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:45.093913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff25acf-f998-4b2d-b206-79efe7fef892 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:45.095335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 749/1519] 10.222.0.13 () {54 vars in 961 bytes} [Tue Oct 4 10:18:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:45.100853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a97d-d592-4993-a317-4a8874327172 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:45.101257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a97d-d592-4993-a317-4a8874327172 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:45.101388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3a97d-d592-4993-a317-4a8874327172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:45.120959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-04a3a97d-d592-4993-a317-4a8874327172 None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:45.122146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-04a3a97d-d592-4993-a317-4a8874327172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:45.131087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 771/1520] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Oct 4 10:18:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:18:45.138753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff25acf-f998-4b2d-b206-79efe7fef892 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:45.151904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff25acf-f998-4b2d-b206-79efe7fef892 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:45.156684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33602a5b-5081-4033-a2dd-d7e45a3fd346 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:45.157256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602a5b-5081-4033-a2dd-d7e45a3fd346 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:45.157528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602a5b-5081-4033-a2dd-d7e45a3fd346 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:45.157794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33602a5b-5081-4033-a2dd-d7e45a3fd346 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:45.166354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602a5b-5081-4033-a2dd-d7e45a3fd346 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:45.167901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33602a5b-5081-4033-a2dd-d7e45a3fd346 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:45.177804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 750/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:18:47.551917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d590ab8-0829-44b9-a36e-039945671ede None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.552443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d590ab8-0829-44b9-a36e-039945671ede None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.552443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d590ab8-0829-44b9-a36e-039945671ede None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.553739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 772/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:18:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:47.559904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991810f-694e-4ef8-801e-45ab42d46e26 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.560350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991810f-694e-4ef8-801e-45ab42d46e26 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.560442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1991810f-694e-4ef8-801e-45ab42d46e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.579506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1991810f-694e-4ef8-801e-45ab42d46e26 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:47.581239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1991810f-694e-4ef8-801e-45ab42d46e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:47.589118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 751/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:18:47.746051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d590ab8-0829-44b9-a36e-039945671ede None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:47.760606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d590ab8-0829-44b9-a36e-039945671ede None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:47.766018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debbd289-d82e-4e6e-9a2d-90771803aade service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:47.766636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbd289-d82e-4e6e-9a2d-90771803aade service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:47.766911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbd289-d82e-4e6e-9a2d-90771803aade service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:47.767178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debbd289-d82e-4e6e-9a2d-90771803aade service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:47.775597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbd289-d82e-4e6e-9a2d-90771803aade service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:47.777160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debbd289-d82e-4e6e-9a2d-90771803aade service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:47.786767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 773/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:18:48.858772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dede9-3d17-4096-aa11-432c35166c4e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.859326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dede9-3d17-4096-aa11-432c35166c4e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.859326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98dede9-3d17-4096-aa11-432c35166c4e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.860647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 752/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Oct 4 10:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:48.867512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181cf6-a413-4786-a976-2aadb6705391 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:48.867954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181cf6-a413-4786-a976-2aadb6705391 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:48.868046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58181cf6-a413-4786-a976-2aadb6705391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:48.887311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-58181cf6-a413-4786-a976-2aadb6705391 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:48.888437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-58181cf6-a413-4786-a976-2aadb6705391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:48.896196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 774/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Oct 4 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:18:49.050070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dede9-3d17-4096-aa11-432c35166c4e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:49.063941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98dede9-3d17-4096-aa11-432c35166c4e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:49.069355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e000c3-f4f3-4d76-be7d-1183ca71af24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:49.069937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e000c3-f4f3-4d76-be7d-1183ca71af24 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:49.070230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e000c3-f4f3-4d76-be7d-1183ca71af24 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:49.070503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e000c3-f4f3-4d76-be7d-1183ca71af24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:49.078153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e000c3-f4f3-4d76-be7d-1183ca71af24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:49.079733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e000c3-f4f3-4d76-be7d-1183ca71af24 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:49.089207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 753/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:18:53.172358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26aca1-f691-46f1-b343-0d5a2f64cd14 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.172868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26aca1-f691-46f1-b343-0d5a2f64cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.172868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d26aca1-f691-46f1-b343-0d5a2f64cd14 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.174319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 775/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:18:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:53.179543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c724dc-4703-49a2-882b-a61405d9c04a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.179946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c724dc-4703-49a2-882b-a61405d9c04a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.180100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c724dc-4703-49a2-882b-a61405d9c04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.200126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-21c724dc-4703-49a2-882b-a61405d9c04a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:53.201313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-21c724dc-4703-49a2-882b-a61405d9c04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:53.210109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 754/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:18:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:18:53.493440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26aca1-f691-46f1-b343-0d5a2f64cd14 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:53.495178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7d26aca1-f691-46f1-b343-0d5a2f64cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:53.575160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d26aca1-f691-46f1-b343-0d5a2f64cd14 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:53.578722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b5e79-66f0-42f1-95b8-280e78130463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:53.579327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5e79-66f0-42f1-95b8-280e78130463 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:53.579602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5e79-66f0-42f1-95b8-280e78130463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:53.579877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b5e79-66f0-42f1-95b8-280e78130463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:53.589312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5e79-66f0-42f1-95b8-280e78130463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:53.590784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b5e79-66f0-42f1-95b8-280e78130463 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:53.600682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 776/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:18:55.733159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9175b-b6ac-45dd-8efa-831d4667aec6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:55.733738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9175b-b6ac-45dd-8efa-831d4667aec6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:55.733738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9175b-b6ac-45dd-8efa-831d4667aec6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:55.735135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 755/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:18:55.740273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45c448-3fe3-4959-b619-df39b643448b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:55.740678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45c448-3fe3-4959-b619-df39b643448b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:55.740812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe45c448-3fe3-4959-b619-df39b643448b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:55.760885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fe45c448-3fe3-4959-b619-df39b643448b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:18:55.762070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fe45c448-3fe3-4959-b619-df39b643448b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:55.770760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 777/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:18:56.058543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9175b-b6ac-45dd-8efa-831d4667aec6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:18:56.060498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-18e9175b-b6ac-45dd-8efa-831d4667aec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:18:56.074047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9175b-b6ac-45dd-8efa-831d4667aec6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:18:56.077571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e51796-0daf-4132-869f-f741153062d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:18:56.078164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e51796-0daf-4132-869f-f741153062d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:18:56.078477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e51796-0daf-4132-869f-f741153062d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:18:56.078781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e51796-0daf-4132-869f-f741153062d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:18:56.087461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e51796-0daf-4132-869f-f741153062d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:18:56.088952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e51796-0daf-4132-869f-f741153062d4 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:18:56.098406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 756/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:18:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:19:00.165793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af1e0-3e9a-44de-bee0-3e242e9fedc2 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:00.166403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af1e0-3e9a-44de-bee0-3e242e9fedc2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:00.166403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08af1e0-3e9a-44de-bee0-3e242e9fedc2 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:00.167714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 778/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:00.172810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02f420-66e9-4bc7-970d-f18007a634ba None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:00.173259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02f420-66e9-4bc7-970d-f18007a634ba None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:00.173372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff02f420-66e9-4bc7-970d-f18007a634ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:00.193078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ff02f420-66e9-4bc7-970d-f18007a634ba None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:00.194289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ff02f420-66e9-4bc7-970d-f18007a634ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:00.202038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 757/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:19:00.510080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af1e0-3e9a-44de-bee0-3e242e9fedc2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:00.511780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f08af1e0-3e9a-44de-bee0-3e242e9fedc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:00.574863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:00.589060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08af1e0-3e9a-44de-bee0-3e242e9fedc2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:00.592673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a6bbe9-3a6f-454f-bff0-4997c67a171c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:00.593546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6bbe9-3a6f-454f-bff0-4997c67a171c service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:00.593826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6bbe9-3a6f-454f-bff0-4997c67a171c service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:00.594095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a6bbe9-3a6f-454f-bff0-4997c67a171c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:00.604046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6bbe9-3a6f-454f-bff0-4997c67a171c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:00.605477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a6bbe9-3a6f-454f-bff0-4997c67a171c service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:00.615807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 779/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:19:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:19:02.735023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e004-fd44-429a-875b-a09793e26ed6 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.735571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e004-fd44-429a-875b-a09793e26ed6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.735571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1617e004-fd44-429a-875b-a09793e26ed6 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.736995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 758/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Oct 4 10:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:19:02.742059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde35ab-b377-4673-a58d-f44b4c9452be None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:02.742479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde35ab-b377-4673-a58d-f44b4c9452be None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:02.742640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bde35ab-b377-4673-a58d-f44b4c9452be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:02.761577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0bde35ab-b377-4673-a58d-f44b4c9452be None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:19:02.762783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0bde35ab-b377-4673-a58d-f44b4c9452be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:02.770917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 780/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Oct 4 10:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:19:03.070716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e004-fd44-429a-875b-a09793e26ed6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:19:03.072389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1617e004-fd44-429a-875b-a09793e26ed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:19:03.085923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1617e004-fd44-429a-875b-a09793e26ed6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:19:03.089435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125c4924-93ce-4aa0-94b1-137b82e03bc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:19:03.090052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4924-93ce-4aa0-94b1-137b82e03bc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:19:03.090364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4924-93ce-4aa0-94b1-137b82e03bc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:19:03.090639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125c4924-93ce-4aa0-94b1-137b82e03bc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:19:03.099722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4924-93ce-4aa0-94b1-137b82e03bc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:19:03.101121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125c4924-93ce-4aa0-94b1-137b82e03bc4 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:19:03.110225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 759/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:19:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:19:12.756895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=11 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:19:13.106168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:28.741892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:28.744109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:28.744109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:28.772788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:28.774004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:28.790370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 781/1540] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:28.799175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:28.799736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:28.799736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:28.828936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:28.830937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:28.844896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 760/1541] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:28.850316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:28.872234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a306536-2b64-43b1-a7bc-42b807bef78f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:28.882170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:28.882878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:28.883107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:28.883386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:28.884049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:28.884353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:28.885969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:28.900202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 782/1542] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:28.908736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:28.929589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d26215-68ca-43f3-bef7-bb17276cb289 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:28.942387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:28.943088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:28.943368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:28.943642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:28.944986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:28.947210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.016008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 761/1543] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:28 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:29.020942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.037081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13c18b2-9d65-4055-a570-6515a58aac54 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.146620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.147254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.147564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.147840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.148484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.150447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.198074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 783/1544] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/users => generated 288 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:20:29.202154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.222534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a332c91-46d8-40bf-811c-a253685c3c15 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.238300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.239345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.239809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.240278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.241417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.243353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.262443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 762/1545] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:29.269290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.289872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a156761-67f3-47e4-b842-8fde8ea57688 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.294910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b8b120-3656-489a-abea-277b4b84e879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.295534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b120-3656-489a-abea-277b4b84e879 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.295814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b120-3656-489a-abea-277b4b84e879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.296087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b8b120-3656-489a-abea-277b4b84e879 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.296740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b120-3656-489a-abea-277b4b84e879 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.298455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b8b120-3656-489a-abea-277b4b84e879 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.316290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 784/1546] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:29.321374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.336793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737fb64-f724-4393-b6e1-2bec94903808 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.343891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea63e38-2b8e-40ca-9e1f-f461b4db371d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.344624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea63e38-2b8e-40ca-9e1f-f461b4db371d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.344910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea63e38-2b8e-40ca-9e1f-f461b4db371d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.345189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea63e38-2b8e-40ca-9e1f-f461b4db371d admin admin] PATH_INFO: `/v3/projects/64ddf0d073bd420c90de975012dc2b9e/users/ef09a3d1aed344eeb6a5100ddd420e16/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.369752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea63e38-2b8e-40ca-9e1f-f461b4db371d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64ddf0d073bd420c90de975012dc2b9e, user_id=ef09a3d1aed344eeb6a5100ddd420e16, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.372620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea63e38-2b8e-40ca-9e1f-f461b4db371d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.379069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b45358-7ff8-493d-ad4b-6eab92a7bdbd None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.379357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b45358-7ff8-493d-ad4b-6eab92a7bdbd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.379639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b45358-7ff8-493d-ad4b-6eab92a7bdbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.405528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 763/1547] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:29 2022] PUT /identity/v3/projects/64ddf0d073bd420c90de975012dc2b9e/users/ef09a3d1aed344eeb6a5100ddd420e16/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:29.412539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-71b45358-7ff8-493d-ad4b-6eab92a7bdbd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.414562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d2e6-773a-4797-bf12-77712e4d0ce7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.414703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-71b45358-7ff8-493d-ad4b-6eab92a7bdbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.414906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d2e6-773a-4797-bf12-77712e4d0ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.415107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1d2e6-773a-4797-bf12-77712e4d0ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.454915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-14c1d2e6-773a-4797-bf12-77712e4d0ce7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.456539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-14c1d2e6-773a-4797-bf12-77712e4d0ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.487908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 785/1548] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.489233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c11683-b9b3-4a4d-b98b-2970a86c8d7e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.489553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c11683-b9b3-4a4d-b98b-2970a86c8d7e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.489882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c11683-b9b3-4a4d-b98b-2970a86c8d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.509733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-89c11683-b9b3-4a4d-b98b-2970a86c8d7e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.511115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-89c11683-b9b3-4a4d-b98b-2970a86c8d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.520864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 786/1549] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.522136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac11fe-d30b-4bdd-bfd7-06a12e3bafa5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.531756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac11fe-d30b-4bdd-bfd7-06a12e3bafa5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.532065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac11fe-d30b-4bdd-bfd7-06a12e3bafa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.559073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1bac11fe-d30b-4bdd-bfd7-06a12e3bafa5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.559734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 764/1550] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.561237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8363272-4ec9-4da5-8278-d17c4eb6b728 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.561514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1bac11fe-d30b-4bdd-bfd7-06a12e3bafa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.561718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8363272-4ec9-4da5-8278-d17c4eb6b728 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.562283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8363272-4ec9-4da5-8278-d17c4eb6b728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.572241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 787/1551] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.573476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.573762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.574090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.587559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a8363272-4ec9-4da5-8278-d17c4eb6b728 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.588769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a8363272-4ec9-4da5-8278-d17c4eb6b728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.593956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.595163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.597372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 765/1552] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.598586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1fee4-26bf-4eaa-aa78-4bcdc3bb87e5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.598874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1fee4-26bf-4eaa-aa78-4bcdc3bb87e5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.599152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d1fee4-26bf-4eaa-aa78-4bcdc3bb87e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.605183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 788/1553] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.605981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.622197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-23d1fee4-26bf-4eaa-aa78-4bcdc3bb87e5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.623424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-23d1fee4-26bf-4eaa-aa78-4bcdc3bb87e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.636160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 766/1554] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.637564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bce23-863f-4559-8c3e-dfedd3ffcb19 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.637858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bce23-863f-4559-8c3e-dfedd3ffcb19 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.638136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711bce23-863f-4559-8c3e-dfedd3ffcb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.651090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:29.657946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-711bce23-863f-4559-8c3e-dfedd3ffcb19 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.659165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-711bce23-863f-4559-8c3e-dfedd3ffcb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.665336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73413325-fbfb-4d56-bb86-42783b0d6b0c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.666953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 767/1555] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.668184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6458603-6cd5-445f-9182-247f145a1a24 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.668468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6458603-6cd5-445f-9182-247f145a1a24 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.668738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6458603-6cd5-445f-9182-247f145a1a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.669182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5be465-5a97-4b97-965e-694826655522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.669894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5be465-5a97-4b97-965e-694826655522 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.670299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5be465-5a97-4b97-965e-694826655522 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.670512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5be465-5a97-4b97-965e-694826655522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.680675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5be465-5a97-4b97-965e-694826655522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.682112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5be465-5a97-4b97-965e-694826655522 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.691379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f6458603-6cd5-445f-9182-247f145a1a24 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.692570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 789/1556] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:29.692795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f6458603-6cd5-445f-9182-247f145a1a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.693827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f794b-27cc-4d24-9f80-c20b4ea816e8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.694144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f794b-27cc-4d24-9f80-c20b4ea816e8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.694447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72f794b-27cc-4d24-9f80-c20b4ea816e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.708315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 768/1557] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.708315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.708315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.708315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.714988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c72f794b-27cc-4d24-9f80-c20b4ea816e8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.716186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c72f794b-27cc-4d24-9f80-c20b4ea816e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.726924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.728119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.730978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 790/1558] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.732514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7c391-b608-4b07-9273-3a60bf79ee61 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.732859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7c391-b608-4b07-9273-3a60bf79ee61 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.733139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc7c391-b608-4b07-9273-3a60bf79ee61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.744997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 769/1559] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.744997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.754879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ebc7c391-b608-4b07-9273-3a60bf79ee61 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.756073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ebc7c391-b608-4b07-9273-3a60bf79ee61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.758979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c05bf25-b52c-4f91-8e35-46943ed91a06 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.763701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.764304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.764576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.764903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.765481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:29.765777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.766808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:20:29.766808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:20:29.766808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:20:29.771795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 791/1560] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.771968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d54209a-d00c-471e-b309-7eea8a7bc85d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.772072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb42ac-addd-4167-b7c6-c40936de3e9a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.772072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb42ac-addd-4167-b7c6-c40936de3e9a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.772072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbb42ac-addd-4167-b7c6-c40936de3e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.783277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 770/1561] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:29.784990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.785345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.785685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.792601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3cbb42ac-addd-4167-b7c6-c40936de3e9a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.793799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3cbb42ac-addd-4167-b7c6-c40936de3e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.805527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 792/1562] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.809197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.809857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.809954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.810056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.810056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.819424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 771/1563] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.820203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.829156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:29.830567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:29.830736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766b2f8e-c130-4d76-ad98-6867be05cb41 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.837202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.838051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.838415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.838748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.839451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:29.839813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.841654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.849647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 793/1564] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:29.851296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.870073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 772/1565] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:29.870073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.870749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37e05ab-7636-4ccb-9c9c-5e2b461fd781 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.871669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.872513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.873137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.873642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.875014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:29.875405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.877584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.884122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9a9fb1-133c-4c81-9849-597ef20fff96 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.889198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.889802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.890081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.890381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.891276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 794/1566] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:29.891451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:29.891451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.891971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.893246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.902504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412bd86-6212-42a4-bb5e-4c8609f3f638 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.907585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.915977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 773/1567] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:29.916172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.916289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:20:29.916388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.922184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e18104-c6d8-4fb1-986c-5e9c66a1c021 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:29.928596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:29.929208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:29.929492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:29.929770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:29.930452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:29.930760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:29.932400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:29.935200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:29.955005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 795/1568] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:29.955005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.956890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 774/1569] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:29.958140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:29.972233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3cb715-cf2b-45da-8bac-658c662e3c0d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.039034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fecd3c-5a13-4c7d-811e-e839610cc957 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.043171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.043857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.044043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.044184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.044534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.044652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.044916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.045188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.045411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:30.045772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.045982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:30.046138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.047570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.047766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.071043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 796/1570] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:30.071248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 775/1571] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:30.071384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.071492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.075865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8726f571-60c1-4b0c-ba7a-c80f10c9c99a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.081447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.082113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.082486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.082825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.084166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.085278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf12253-c09c-45b3-bebf-b40320f72972 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.085551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.090991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.091667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.092034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.092454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.093327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.097176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.129821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 776/1572] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.130783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.139724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 797/1573] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.140400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.208675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:30.225392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b58c2-2088-4c5e-93a6-e57dec18782a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.233950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.235411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.235891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdce89d-9019-498d-a368-1126fa63c385 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.236161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.237288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.238889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.243287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.244972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.246305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.246637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.246938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.248009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.250458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.285509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 777/1574] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/projects => generated 371 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.285992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 798/1575] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/projects => generated 367 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.286972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.287096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.347235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:30.360136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a54b959-f632-432c-9be9-62633c5822d4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.362769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7cf277-cc09-4834-995d-144c02a169b1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.367312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.368205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.368487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.368764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.368940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.369657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.369800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.369902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.370096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.370726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:20:30.370726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:20:30.370726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:20:30.370958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.371939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.372654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.399497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 778/1576] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.400555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.406859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 799/1577] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/projects => generated 391 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.407736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.417659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec02c5e-28bc-4ef9-9a24-6b9ed4b783e0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.495605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a41a8d-cad9-4ea3-a09e-297ef2142309 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.497919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.498570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.498849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.499125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.499761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.500929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.501668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.501859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.503006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.503353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.504284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.506335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.534880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 800/1578] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 290 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.536067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.549346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 779/1579] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/projects => generated 379 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.550556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.554021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203bd366-fe65-48b8-af4c-ca26dc979577 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.621437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:30.633144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8a721f-a925-4ec1-be91-6275ccd86dce admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.639754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.640633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.640996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.641503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.641895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.642183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.642481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.642601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.643518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.643830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.645625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.646187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.673141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 780/1580] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.674687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.679798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 801/1581] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.681644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.687193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de89bbdd-b1be-4b09-83a6-d12818a95f1d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.695160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.695852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.696186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.696463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.697166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.699374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.700788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefce48-285a-46c3-b0d7-6498bb15c18a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.707938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.708890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.709172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.711324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.712239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.714866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.732687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 802/1582] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.732687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.733698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 781/1583] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 284 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.734972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.745103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0c0e6d-4b5d-4179-920b-9ca7222b0910 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.746819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d7b015-df15-48e3-9210-d177d458022f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.750721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.751765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.752695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.753383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.753538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.754309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.754779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.755248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.755715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.756550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.757131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.759216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.774510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 782/1584] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.775780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.789454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cdf3d3d-1fb3-4a1e-a79a-1891f37487d5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.795734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 803/1585] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 296 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.795734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.800740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.801827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.802411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.803023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.803878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.806250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.807666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f79e30-6d45-44f2-ac1f-ac6b01dfb6b9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.813124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.813907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.814186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.815756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.816537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.818618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.838797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 804/1586] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.838797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.841843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 783/1587] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:30 2022] POST /identity/v3/users => generated 290 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:30.843418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.853910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991b36c-a9e7-41c0-adaa-e93eafdd8092 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.857031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945a6a72-ccbe-4394-9b93-d5f932fea64d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.859779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.860542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.860820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.861619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.862532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.863920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.864767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.865252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.865375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.865856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.866756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.869178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.878292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:30.886864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 805/1588] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.886864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.891404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 784/1589] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.892250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.902300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8721f783-9718-4298-88e0-23a7062a7a4a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.902518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbe5b3f-0347-4e33-9bec-fe217e203188 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.907767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.909436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.909436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.909645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.910516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.912197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.922997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.923643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.924090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.924393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.925009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 785/1590] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.925129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.925709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.926716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.936711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a3496a-9361-40fd-ae22-18ff39c1c9a3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.942725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 806/1591] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.942725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.945900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.947716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.948054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.949495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.950220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.952117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.958706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130deff9-ac98-43d1-bc49-aeae68d51b38 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.970309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 786/1592] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.970309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.970607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.971313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:30.971313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:30.971499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:30.974929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:30.976555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:30.986930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f8506-c790-4612-9cfc-23457206083f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:30.992866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 807/1593] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:30.993712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:30.994899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:30.999138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.000418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.000634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.003103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.004772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286020-3a49-483e-a21d-392cdb783c2a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.006434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.009883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.010568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.010849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.011124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.012138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.014331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.025333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 808/1594] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:31.026179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.031328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 787/1595] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:30 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:31.036014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.037629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5725b-459f-43d2-93d7-8e89ced59fbb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.045266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.046026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.046422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.046789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] PATH_INFO: `/v3/projects/4360ce98d2db41fb90c378f2df1d653e/users/3bb0f21252f34df18d1c932e56687aa2/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.048149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f469f6-cfef-4f80-83db-ec0b9495d403 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.062938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.064083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.064237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4360ce98d2db41fb90c378f2df1d653e, user_id=3bb0f21252f34df18d1c932e56687aa2, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.064449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.065738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] PATH_INFO: `/v3/projects/0150cb46bfdf4f50baf0ef3ecba77695/users/7c9b4978f33247aea4d5768d98120e71/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.066164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:20:31.066164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:20:31.066164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:20:31.068431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.085586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 809/1596] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/4360ce98d2db41fb90c378f2df1d653e/users/3bb0f21252f34df18d1c932e56687aa2/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:31.094488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] RBAC: Authorizing `identity:create_grant(project_id=0150cb46bfdf4f50baf0ef3ecba77695, user_id=7c9b4978f33247aea4d5768d98120e71, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.094651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.094750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.113668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 788/1597] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/0150cb46bfdf4f50baf0ef3ecba77695/users/7c9b4978f33247aea4d5768d98120e71/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:31.120955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.234505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8f7502-9469-497b-87f6-6a8719289c54 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.236627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec116fe-bc52-4f8c-90ef-5ceee415264b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.241539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.242230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.242670 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.243006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.243828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.244826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.245587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.246161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.246598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.246971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.247741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.249446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.263830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 810/1598] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:31.263830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.265024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 789/1599] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:31.265935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.281967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b429ac4-0209-4068-85df-f6eabfdfffe7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.289175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.290355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.290638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.290910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] PATH_INFO: `/v3/projects/50286c4e105445e99ed16d1d46fd1a12/users/f2c5d668d9754c4bad54a83efae9c877/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.293604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd1d54d-8f7d-4881-8602-23ba43a282fe admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.299912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4cec67-a82d-435c-948b-1dde43f0d188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.301781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cec67-a82d-435c-948b-1dde43f0d188 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.302176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cec67-a82d-435c-948b-1dde43f0d188 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.303337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4cec67-a82d-435c-948b-1dde43f0d188 admin admin] PATH_INFO: `/v3/projects/9a791fa3fed340eca53f87f8caee245e/users/201776c863f14727b3bb7b7912e89f46/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.315553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50286c4e105445e99ed16d1d46fd1a12, user_id=f2c5d668d9754c4bad54a83efae9c877, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.320081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.327868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cec67-a82d-435c-948b-1dde43f0d188 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a791fa3fed340eca53f87f8caee245e, user_id=201776c863f14727b3bb7b7912e89f46, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.338492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4cec67-a82d-435c-948b-1dde43f0d188 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.343356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 811/1600] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/50286c4e105445e99ed16d1d46fd1a12/users/f2c5d668d9754c4bad54a83efae9c877/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:31.346705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.366841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:31.368222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 790/1601] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/9a791fa3fed340eca53f87f8caee245e/users/201776c863f14727b3bb7b7912e89f46/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:31.378486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6f55-0cb1-4a3e-8ebd-71ee29766c39 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.378486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6f55-0cb1-4a3e-8ebd-71ee29766c39 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.378486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c6f55-0cb1-4a3e-8ebd-71ee29766c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.410161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-232c6f55-0cb1-4a3e-8ebd-71ee29766c39 None None] MFA Rules not processed for user `3bb0f21252f34df18d1c932e56687aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:31.411829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-232c6f55-0cb1-4a3e-8ebd-71ee29766c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:31.453525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45af8cba-e401-4ad7-a886-c9b7e9db4fa6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.515991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 791/1602] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:31.517938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.518490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.519578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.531155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:31.545086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.546155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.546776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.547162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.547472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:31.548267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.549317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:31.550443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.563771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 812/1603] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:31 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:31.570261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 792/1604] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:31.570689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.570800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.642225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b68f517-23e6-48e5-b612-b86e9ce4fcf3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.643678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef55e1-54f6-48ae-9590-c71642a4ae07 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.643678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12f09f-d281-42fa-bb6d-d87882df52b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.644131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07f7b1-538b-4c5f-8b95-5f85b7189bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.644237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f09f-d281-42fa-bb6d-d87882df52b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.644237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f09f-d281-42fa-bb6d-d87882df52b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.644403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f7b1-538b-4c5f-8b95-5f85b7189bca admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.644487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12f09f-d281-42fa-bb6d-d87882df52b1 admin admin] PATH_INFO: `/v3/projects/4bcab3ae3d924513aa07f4feea89b2ce/users/2ea5cf0e020a4d60903767f4e693a104/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.644567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f7b1-538b-4c5f-8b95-5f85b7189bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.644567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07f7b1-538b-4c5f-8b95-5f85b7189bca admin admin] PATH_INFO: `/v3/projects/d85cf718f1b248c192c138bea872fc0d/users/28c104b3e1284882b72d6b6933475029/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.645396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f09f-d281-42fa-bb6d-d87882df52b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bcab3ae3d924513aa07f4feea89b2ce, user_id=2ea5cf0e020a4d60903767f4e693a104, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.645396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12f09f-d281-42fa-bb6d-d87882df52b1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.645695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07f7b1-538b-4c5f-8b95-5f85b7189bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85cf718f1b248c192c138bea872fc0d, user_id=28c104b3e1284882b72d6b6933475029, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.645695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07f7b1-538b-4c5f-8b95-5f85b7189bca admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.657050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 813/1605] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/4bcab3ae3d924513aa07f4feea89b2ce/users/2ea5cf0e020a4d60903767f4e693a104/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:31.660167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe411172-5255-45c2-af3a-485889632789 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.660870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe411172-5255-45c2-af3a-485889632789 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.661367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe411172-5255-45c2-af3a-485889632789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.671706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 793/1606] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/d85cf718f1b248c192c138bea872fc0d/users/28c104b3e1284882b72d6b6933475029/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:31.680005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.680293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.680566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.714845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fe411172-5255-45c2-af3a-485889632789 None None] MFA Rules not processed for user `f2c5d668d9754c4bad54a83efae9c877`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:31.716090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fe411172-5255-45c2-af3a-485889632789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:31.729482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] MFA Rules not processed for user `201776c863f14727b3bb7b7912e89f46`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:31.730942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:31.829084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 794/1607] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:31.829793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.852066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 814/1608] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:31.854137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe411172-5255-45c2-af3a-485889632789 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.876876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:31.916466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8e5857-db7c-4004-aac2-c04fef256f46 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.926692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.927916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.928349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.928688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.931515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.933619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.952437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe411172-5255-45c2-af3a-485889632789 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:31.958081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e93bc-57f3-436f-ae16-f90109d2583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:31.959463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e93bc-57f3-436f-ae16-f90109d2583f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:31.959463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e93bc-57f3-436f-ae16-f90109d2583f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:31.959463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e93bc-57f3-436f-ae16-f90109d2583f admin admin] PATH_INFO: `/v3/projects/6c286d36379c4275b5d533acd2431502/users/6dc8e0d5cbb347eb8cbce3c49fda452d/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:31.970491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 795/1609] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:31 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:31.970491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:31.976423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e93bc-57f3-436f-ae16-f90109d2583f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c286d36379c4275b5d533acd2431502, user_id=6dc8e0d5cbb347eb8cbce3c49fda452d, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:31.979615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973e93bc-57f3-436f-ae16-f90109d2583f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:31.999151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 815/1610] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:31 2022] PUT /identity/v3/projects/6c286d36379c4275b5d533acd2431502/users/6dc8e0d5cbb347eb8cbce3c49fda452d/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:32.000659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37ba1-6d22-41f7-8dd9-9e921c235acc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.001067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37ba1-6d22-41f7-8dd9-9e921c235acc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.001446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f37ba1-6d22-41f7-8dd9-9e921c235acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.032752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-01f37ba1-6d22-41f7-8dd9-9e921c235acc None None] MFA Rules not processed for user `2ea5cf0e020a4d60903767f4e693a104`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:32.034469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-01f37ba1-6d22-41f7-8dd9-9e921c235acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:32.039326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:32.043507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79f7f6-e586-4a79-89fa-5faf0781dc4c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.103587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cadbbd-2bcb-4110-b04c-6bf1bd1d6be0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.104808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cadbbd-2bcb-4110-b04c-6bf1bd1d6be0 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.105159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cadbbd-2bcb-4110-b04c-6bf1bd1d6be0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.105509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cadbbd-2bcb-4110-b04c-6bf1bd1d6be0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.109531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 816/1611] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:32.111827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e9eb-5abd-4512-bb70-1996fdfeadbe None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.112917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e9eb-5abd-4512-bb70-1996fdfeadbe None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.113376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08e9eb-5abd-4512-bb70-1996fdfeadbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.136529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7e08e9eb-5abd-4512-bb70-1996fdfeadbe None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:32.138400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7e08e9eb-5abd-4512-bb70-1996fdfeadbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:32.184837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cadbbd-2bcb-4110-b04c-6bf1bd1d6be0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.186362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cadbbd-2bcb-4110-b04c-6bf1bd1d6be0 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.214563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 796/1612] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:32.216121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de1d08-0026-46fa-a1d2-ff0f22c46f04 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.216416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de1d08-0026-46fa-a1d2-ff0f22c46f04 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.217781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de1d08-0026-46fa-a1d2-ff0f22c46f04 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.217933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 817/1613] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:32.222893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 797/1614] 10.222.0.13 () {54 vars in 961 bytes} [Tue Oct 4 10:20:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:32.224687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58e84-28f0-444c-9a63-2d5ca0bc8197 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.224687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58e84-28f0-444c-9a63-2d5ca0bc8197 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.224920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da58e84-28f0-444c-9a63-2d5ca0bc8197 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.225078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de1d08-0026-46fa-a1d2-ff0f22c46f04 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.226897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 818/1615] 10.222.0.13 () {54 vars in 961 bytes} [Tue Oct 4 10:20:32 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:32.228130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676995c-efa3-4ca4-98d4-f29e319f36eb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.228505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676995c-efa3-4ca4-98d4-f29e319f36eb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.228871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8676995c-efa3-4ca4-98d4-f29e319f36eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.249685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de1d08-0026-46fa-a1d2-ff0f22c46f04 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.256851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.257527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.257805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.258091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.258808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.260886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.266160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8676995c-efa3-4ca4-98d4-f29e319f36eb None None] MFA Rules not processed for user `6dc8e0d5cbb347eb8cbce3c49fda452d`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:32.267823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8676995c-efa3-4ca4-98d4-f29e319f36eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:32.294397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:32.295460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 798/1616] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:32 2022] POST /identity/v3/users => generated 293 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:32.301516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.398376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 819/1617] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:32.398376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.398376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.398376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.403199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229e41d-26f8-4d1c-9c2b-7c3a634b66f6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.414339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44068a9-668a-4c5e-a9c0-9ddbdb818b82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.415116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44068a9-668a-4c5e-a9c0-9ddbdb818b82 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.415935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44068a9-668a-4c5e-a9c0-9ddbdb818b82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.416229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44068a9-668a-4c5e-a9c0-9ddbdb818b82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.436406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44068a9-668a-4c5e-a9c0-9ddbdb818b82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.438789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44068a9-668a-4c5e-a9c0-9ddbdb818b82 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.461086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 799/1618] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:32.464816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.465100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.465375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.485102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:32.488341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:32.513186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:32.514695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:32.600544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:32.650258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 800/1619] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Oct 4 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:32.651523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.661985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce7393-9d28-4999-a55e-4912be45298f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.666271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.667033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.667318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.667640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.673029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 820/1620] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Oct 4 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:32.674882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.679540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.681217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.695020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 801/1621] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:32.695752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.698765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24632ff4-3e61-4ebf-88de-2e6a36ad7ac3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.710038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a762ec3-8948-4064-a360-afdf77e1e381 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.710636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.711266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.711540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.711807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.712523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.714226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.723401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.724088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.724536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.724790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.725431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.727846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.739428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 821/1622] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:32.740294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.764842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5ff8dc-264e-46aa-8287-4a72438e5d2a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.782741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.783449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 802/1623] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:32 2022] POST /identity/v3/projects => generated 379 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:32.783905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.784176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.784384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.784489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:32.900877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:32.907693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:32.953120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 822/1624] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:32.954231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:32.977380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c54d95-f800-4c80-8e91-f9e9b57ca2e2 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.978671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378039-0c8e-4ded-a429-9d393cb07e52 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:32.990030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:32.990388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:32.990670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:32.990945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.021194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:33.086580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.087349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.088586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.088973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] PATH_INFO: `/v3/projects/6f8d4f5fdf5c4393914bbe2d30cfb855/users/5cda7bc3d33944a9b31e09ff7bdf189d/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.116935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8d4f5fdf5c4393914bbe2d30cfb855, user_id=5cda7bc3d33944a9b31e09ff7bdf189d, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.120741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.138014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.139649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.144594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 823/1625] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:32 2022] PUT /identity/v3/projects/6f8d4f5fdf5c4393914bbe2d30cfb855/users/5cda7bc3d33944a9b31e09ff7bdf189d/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:33.145953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.171536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 803/1626] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:33.172399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.223494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ad0f95-11b8-45a6-89e0-7f3266ae1601 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.261261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3699e9-dc94-43ae-828f-3e8101c89257 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.285739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.286883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.287201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.287484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.288190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.291528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.340916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:33.356595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 824/1627] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:33 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:33.361837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.391989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f27fde1-32d4-4420-a1df-b21f6a494f12 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.409061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.411141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.411437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.411793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.417596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.423088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.436909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.437698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.437983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.438299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.438947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.441947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.463837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 825/1628] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:33.467065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.491904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 804/1629] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:33.492646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebe736-8617-433d-861e-d407f24ae688 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.492910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.504674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.505292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.505581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.509972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] PATH_INFO: `/v3/projects/06d7abc3014648a38ceffeffd1d4f51d/users/2abcbc861c59499ca8758f887de7830e/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.512064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daf72b1-6cbd-4683-aa37-58c113c2687f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.528351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dcb92c-e76b-4f69-927a-d245b2323457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.528556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb92c-e76b-4f69-927a-d245b2323457 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.528977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb92c-e76b-4f69-927a-d245b2323457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.534165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dcb92c-e76b-4f69-927a-d245b2323457 admin admin] PATH_INFO: `/v3/projects/6f8d4f5fdf5c4393914bbe2d30cfb855/users/5cda7bc3d33944a9b31e09ff7bdf189d/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.538012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb92c-e76b-4f69-927a-d245b2323457 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f8d4f5fdf5c4393914bbe2d30cfb855, user_id=5cda7bc3d33944a9b31e09ff7bdf189d, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.540780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dcb92c-e76b-4f69-927a-d245b2323457 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.555335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d7abc3014648a38ceffeffd1d4f51d, user_id=2abcbc861c59499ca8758f887de7830e, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.563227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.594737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 805/1630] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:33 2022] PUT /identity/v3/projects/6f8d4f5fdf5c4393914bbe2d30cfb855/users/5cda7bc3d33944a9b31e09ff7bdf189d/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:33.596840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 826/1631] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:33 2022] PUT /identity/v3/projects/06d7abc3014648a38ceffeffd1d4f51d/users/2abcbc861c59499ca8758f887de7830e/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:33.600568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.600729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.600995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.601282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.641992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] MFA Rules not processed for user `5cda7bc3d33944a9b31e09ff7bdf189d`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:33.644023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:33.823481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 806/1632] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:33 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:20:33.841299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.865426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e5893a-d999-45b3-bfca-0768a834b64d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.883871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f046ed-9b64-4eee-b6ba-6f3b8b2a8058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.884513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f046ed-9b64-4eee-b6ba-6f3b8b2a8058 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.885952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f046ed-9b64-4eee-b6ba-6f3b8b2a8058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.886265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f046ed-9b64-4eee-b6ba-6f3b8b2a8058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.886939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f046ed-9b64-4eee-b6ba-6f3b8b2a8058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:33.888561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f046ed-9b64-4eee-b6ba-6f3b8b2a8058 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:33.902227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:33.938413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 827/1633] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:20:33.948102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4592-ceb8-4601-bfd3-9ca95a6c6f46 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.948398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4592-ceb8-4601-bfd3-9ca95a6c6f46 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.948675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fc4592-ceb8-4601-bfd3-9ca95a6c6f46 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.961083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 828/1634] 10.222.0.13 () {54 vars in 912 bytes} [Tue Oct 4 10:20:33 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:33.964368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b0adbf-609a-41e0-baad-80c3f235cc8f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:33.967007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d903c-5d5f-4c6c-96ff-5802f8b6a2f9 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.967583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d903c-5d5f-4c6c-96ff-5802f8b6a2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.967873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d903c-5d5f-4c6c-96ff-5802f8b6a2f9 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:33.972712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 829/1635] 10.222.0.13 () {54 vars in 912 bytes} [Tue Oct 4 10:20:33 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:33.973558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d903c-5d5f-4c6c-96ff-5802f8b6a2f9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:33.974764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8ded13-9938-458f-a5a4-b53c485cd310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:33.975403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ded13-9938-458f-a5a4-b53c485cd310 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:33.975652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ded13-9938-458f-a5a4-b53c485cd310 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:33.975935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8ded13-9938-458f-a5a4-b53c485cd310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.012394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ded13-9938-458f-a5a4-b53c485cd310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:34.013864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8ded13-9938-458f-a5a4-b53c485cd310 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:34.025928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d903c-5d5f-4c6c-96ff-5802f8b6a2f9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:34.036879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 807/1636] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:20:34.038292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef734e-87a2-4809-b05e-730915d0882e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:34.038575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef734e-87a2-4809-b05e-730915d0882e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:34.038852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdef734e-87a2-4809-b05e-730915d0882e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.050114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acbdf0c-f684-47d0-b5df-f560f84b1e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:34.050708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdf0c-f684-47d0-b5df-f560f84b1e65 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:34.050994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdf0c-f684-47d0-b5df-f560f84b1e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:34.051278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acbdf0c-f684-47d0-b5df-f560f84b1e65 admin admin] PATH_INFO: `/v3/projects/06d7abc3014648a38ceffeffd1d4f51d/users/2abcbc861c59499ca8758f887de7830e/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.059551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbdf0c-f684-47d0-b5df-f560f84b1e65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06d7abc3014648a38ceffeffd1d4f51d, user_id=2abcbc861c59499ca8758f887de7830e, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:34.063566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acbdf0c-f684-47d0-b5df-f560f84b1e65 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:34.087017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cdef734e-87a2-4809-b05e-730915d0882e None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:34.088237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cdef734e-87a2-4809-b05e-730915d0882e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:34.099696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 830/1637] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:33 2022] PUT /identity/v3/projects/06d7abc3014648a38ceffeffd1d4f51d/users/2abcbc861c59499ca8758f887de7830e/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:34.099696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:34.099696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:34.099696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.146628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:34.147901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:34.197550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:34.265466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 808/1638] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:34.270873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:34.271646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:34.271934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.355341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] MFA Rules not processed for user `2abcbc861c59499ca8758f887de7830e`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:34.360674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:34.373234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 831/1639] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:34.376360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:34.512598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a4c6de-6ede-43e9-ba5b-d1c74e9131a3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:34.520757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:34.521850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:34.521850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:34.522038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.544879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 809/1640] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:34 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:34.545787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:34.552543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:34.554159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:34.561345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73aa8b1-366b-4e99-9e00-4a93c13dbeb0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:34.567097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:34.568148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:34.569221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:34.569779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:34.576509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 832/1641] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:34.647131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:34.648811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:34.661382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 810/1642] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:34 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:35.089047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:35.093860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:35.103042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1678d7d0-cf34-4117-b1a4-da1c00cad729 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:35.113453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ad5aa2-e236-4d10-8e6a-d3d25a7cceab service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:35.114929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:35.115744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:35.116311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:35.116808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:35.126064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:35.129224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:35.130738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:35.131150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:35.141772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:35.145119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:35.166906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:35.170771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 833/1643] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:35.173466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:35.227745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 811/1644] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:35.765703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:35.765703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69427f-9b28-4684-82da-e387bf563c47 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:35.783214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b9d41e-6c09-4aab-bb01-f0d00c575c6c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:35.784750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9d41e-6c09-4aab-bb01-f0d00c575c6c service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:35.785060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9d41e-6c09-4aab-bb01-f0d00c575c6c service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:35.785339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b9d41e-6c09-4aab-bb01-f0d00c575c6c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:35.938828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9d41e-6c09-4aab-bb01-f0d00c575c6c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:35.941914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b9d41e-6c09-4aab-bb01-f0d00c575c6c service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:35.966806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 834/1645] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:37.987080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:38.175319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:38.225559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636d7db1-13d6-4576-97f1-7c4bdd778db0 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:38.240338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:38.241252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:38.241760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:38.242290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] PATH_INFO: `/v3/users/f2c5d668d9754c4bad54a83efae9c877` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:38.243761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2c5d668d9754c4bad54a83efae9c877)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:38.246118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:38.391261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2c5d668d9754c4bad54a83efae9c877'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:38.395807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2c5d668d9754c4bad54a83efae9c877'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:38.411440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2c5d668d9754c4bad54a83efae9c877'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:38.412402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2c5d668d9754c4bad54a83efae9c877'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:38.429353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2c5d668d9754c4bad54a83efae9c877'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:38.457923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 812/1646] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:37 2022] DELETE /identity/v3/users/f2c5d668d9754c4bad54a83efae9c877 => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:38.462499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:38.462989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:38.463123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:38.473090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:38.492791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:38.503029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:38.680252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 835/1647] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:38.704387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:38.707322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:38.921296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cb6a78-9b84-494f-9fab-044c7ecc7eee admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:38.929061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8febc49-6218-4098-bdf3-aa66aea73962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:38.929730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8febc49-6218-4098-bdf3-aa66aea73962 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:38.930053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8febc49-6218-4098-bdf3-aa66aea73962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:38.931076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8febc49-6218-4098-bdf3-aa66aea73962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:38.951862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8febc49-6218-4098-bdf3-aa66aea73962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:38.957546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8febc49-6218-4098-bdf3-aa66aea73962 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:38.972788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 813/1648] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:40.034045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:40.035601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:40.055205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ad6dcb-c59b-4a29-b5eb-d4b06d19310c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:40.061499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:40.062423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:40.062951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:40.063371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] PATH_INFO: `/v3/projects/50286c4e105445e99ed16d1d46fd1a12` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:40.065202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=50286c4e105445e99ed16d1d46fd1a12)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:40.067563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:40.162661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50286c4e105445e99ed16d1d46fd1a12', 'request_id': 'req-f18232db-8946-40aa-9430-9a50995cdb2c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:40.174674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50286c4e105445e99ed16d1d46fd1a12', 'request_id': 'req-f18232db-8946-40aa-9430-9a50995cdb2c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:40.191106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50286c4e105445e99ed16d1d46fd1a12', 'request_id': 'req-f18232db-8946-40aa-9430-9a50995cdb2c'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:40.213921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] The token cache is being invalidate because project 50286c4e105445e99ed16d1d46fd1a12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:20:40.214477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:40.218316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f18232db-8946-40aa-9430-9a50995cdb2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:40.223596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 836/1649] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:20:40 2022] DELETE /identity/v3/projects/50286c4e105445e99ed16d1d46fd1a12 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:40.228396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7fd58-e9c7-4adb-abad-a3903b6c60df None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:40.228897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7fd58-e9c7-4adb-abad-a3903b6c60df None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:40.228992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a7fd58-e9c7-4adb-abad-a3903b6c60df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:40.285879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-92a7fd58-e9c7-4adb-abad-a3903b6c60df None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:40.290750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-92a7fd58-e9c7-4adb-abad-a3903b6c60df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:40.520953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:40.521240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:40.521518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:40.544332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 814/1650] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:40.549307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:40.549594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:40.549891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:40.629361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:40.638975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:40.660127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] MFA Rules not processed for user `ef09a3d1aed344eeb6a5100ddd420e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:40.666697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:40.688120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 815/1651] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:40.692644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:40.697019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:40.707745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638c9476-d6f5-4e8a-8a7d-74c0c13b5eeb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:40.722146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:40.731136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:40.731555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:40.731834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:40.732755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:40.733061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:40.734777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:40.773238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 816/1652] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:40.779818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:40.805238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17059cc-10a9-45bb-ad04-3620d342fff5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:40.817528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:40.818537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:40.819026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:40.819493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:40.820342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:40.822497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:40.899776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 817/1653] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:40 2022] POST /identity/v3/projects => generated 403 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:40.909869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:40.911892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 837/1654] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:20:40.919122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:40.924146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:40.966446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:41.015113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605fc17c-9f7f-4ebb-9cf5-6b9efaef4a55 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:41.030841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:41.031469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.031886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.032186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.033457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:41.035841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:41.092618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 818/1655] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:40 2022] POST /identity/v3/users => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:41.098315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:41.120803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b943c52b-eef1-4f0a-aa18-aff95357d690 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:41.122464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad955cc0-1a9c-49ac-8fe5-70371fa6b879 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:41.134007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:41.134568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.134860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.136515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.137200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:41.149243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:41.185425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 819/1656] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:41.190010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:41.237985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e2921-8f34-4590-b822-53d2c307055e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:41.246508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:41.247542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.248057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.248514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.249343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:41.251300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:41.278415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 820/1657] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:41.281726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:41.294184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:41.295880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.296646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.297138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.304474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eec1249-b068-4cae-8b80-543f5f38277c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:41.318522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831f608-f40c-4ed0-babb-a6bbf0a1f6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:41.318983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f608-f40c-4ed0-babb-a6bbf0a1f6ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.319197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f608-f40c-4ed0-babb-a6bbf0a1f6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.319463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831f608-f40c-4ed0-babb-a6bbf0a1f6ae admin admin] PATH_INFO: `/v3/projects/97519cdb8d864cf9b5f223f675fac2ac/users/3d4a02d754dd449fb4fc81cd34620c1a/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.356059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831f608-f40c-4ed0-babb-a6bbf0a1f6ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=97519cdb8d864cf9b5f223f675fac2ac, user_id=3d4a02d754dd449fb4fc81cd34620c1a, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:41.360323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831f608-f40c-4ed0-babb-a6bbf0a1f6ae admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:41.396031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 821/1658] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:41 2022] PUT /identity/v3/projects/97519cdb8d864cf9b5f223f675fac2ac/users/3d4a02d754dd449fb4fc81cd34620c1a/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:41.399551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3f32-8e88-4d93-acd5-ddfb183794ea None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.401433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3f32-8e88-4d93-acd5-ddfb183794ea None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.401433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3f32-8e88-4d93-acd5-ddfb183794ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.442682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8b1b3f32-8e88-4d93-acd5-ddfb183794ea None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:41.444412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:41.446137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:41.447178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8b1b3f32-8e88-4d93-acd5-ddfb183794ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:41.464398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 838/1659] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:20:41.552051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 822/1660] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:41.566774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:41.568290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:41.743725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f796644-6c5f-45a5-b8ec-aa5651693f74 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:41.753104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:41.856880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c52ef-1995-4e1b-9b83-7c40404fe48d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:41.857699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c52ef-1995-4e1b-9b83-7c40404fe48d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:41.857978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c52ef-1995-4e1b-9b83-7c40404fe48d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:41.858561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c52ef-1995-4e1b-9b83-7c40404fe48d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:41.883234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c52ef-1995-4e1b-9b83-7c40404fe48d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:41.886113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5c52ef-1995-4e1b-9b83-7c40404fe48d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:41.912425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 839/1661] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:42.609016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:42.610058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:42.610275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:42.696111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:42.723826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] MFA Rules not processed for user `28c104b3e1284882b72d6b6933475029`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:42.729248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:42.956716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 823/1662] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:42.967551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078bd54-7b98-4a5f-8467-531befca46a7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:42.968001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078bd54-7b98-4a5f-8467-531befca46a7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:42.968171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8078bd54-7b98-4a5f-8467-531befca46a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:43.006822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8078bd54-7b98-4a5f-8467-531befca46a7 None None] MFA Rules not processed for user `28c104b3e1284882b72d6b6933475029`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:43.008514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8078bd54-7b98-4a5f-8467-531befca46a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:43.017935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 840/1663] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:43.029667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:43.191959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce126fc-5c47-4254-9aec-25a6b5997c6b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:43.212073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888e30dd-bde8-4686-8d02-d77071fdeedb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:43.212809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e30dd-bde8-4686-8d02-d77071fdeedb service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:43.213099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e30dd-bde8-4686-8d02-d77071fdeedb service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:43.213390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888e30dd-bde8-4686-8d02-d77071fdeedb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:43.225907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e30dd-bde8-4686-8d02-d77071fdeedb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:43.227664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888e30dd-bde8-4686-8d02-d77071fdeedb service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:43.250098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 824/1664] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:44.091836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cf9b-f2ef-4fbf-b124-b15bf45e323e None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:44.093985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cf9b-f2ef-4fbf-b124-b15bf45e323e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:44.094665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e1cf9b-f2ef-4fbf-b124-b15bf45e323e None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:44.098048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 841/1665] 10.222.0.13 () {54 vars in 912 bytes} [Tue Oct 4 10:20:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:44.103744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755ba30-c688-47b0-a314-be3ebd79ae58 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:44.104242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755ba30-c688-47b0-a314-be3ebd79ae58 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:44.104498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755ba30-c688-47b0-a314-be3ebd79ae58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:44.218865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7755ba30-c688-47b0-a314-be3ebd79ae58 None None] MFA Rules not processed for user `0e568115c67243d3904359b84e0a790e`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:44.226048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7755ba30-c688-47b0-a314-be3ebd79ae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:44.252338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 825/1666] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:20:44 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:44.287289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1cf9b-f2ef-4fbf-b124-b15bf45e323e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:44.291445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-88e1cf9b-f2ef-4fbf-b124-b15bf45e323e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:44.341711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e1cf9b-f2ef-4fbf-b124-b15bf45e323e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:44.358291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:44.359225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:44.359837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:44.360304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:44.381213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:44.387283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:44.413366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 842/1667] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:44.865373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:44.865697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:44.865985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:44.958057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] MFA Rules not processed for user `3d4a02d754dd449fb4fc81cd34620c1a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:44.959284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:45.110496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:45.137307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 826/1668] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:44 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:45.142686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:45.174715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fc88db-b18c-4fc4-a498-c54ab8a34eae service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:45.197738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:45.198615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:45.199380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:45.201305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:45.201305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:45.208509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:45.302501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:45.303606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 843/1669] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:45 2022] POST /identity/v3/projects => generated 403 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:45.321919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf22c1-668b-459f-8f31-6fb7e2277b9a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:45.406927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:45.410772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:45.555095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af39b3e1-ed1c-49ba-a4da-dc5c69ee9b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:45.555734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39b3e1-ed1c-49ba-a4da-dc5c69ee9b52 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:45.556851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39b3e1-ed1c-49ba-a4da-dc5c69ee9b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:45.557134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af39b3e1-ed1c-49ba-a4da-dc5c69ee9b52 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:45.557788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39b3e1-ed1c-49ba-a4da-dc5c69ee9b52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:45.561816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af39b3e1-ed1c-49ba-a4da-dc5c69ee9b52 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:45.615292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be89ffe-7104-4127-abfd-8528949c82b0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:45.620544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 827/1670] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:45 2022] POST /identity/v3/users => generated 308 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:45.622755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:45.623312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:45.623810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:45.624286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:45.625029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:45.625361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:45.625641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:45.628375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:45.645195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:45.649416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:45.669052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] MFA Rules not processed for user `28c104b3e1284882b72d6b6933475029`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:45.671743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 844/1671] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:20:45.673121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:45.689012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:45.689492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc34f5-8552-4690-aefc-c8264e0faa10 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:45.695464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:45.696275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:45.696797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:45.697662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:45.697999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:45.700100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:45.714926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 845/1672] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:45.719649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:45.733864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e62049-fcfd-44ff-96f8-c34770c36aab admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:45.739701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:45.740511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:45.740923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:45.741391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] PATH_INFO: `/v3/projects/c2588fe3d23f434d8d329da0e3028d74/users/6b373accd7804d19927cb077925444c3/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:45.760064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2588fe3d23f434d8d329da0e3028d74, user_id=6b373accd7804d19927cb077925444c3, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:45.763136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:45.796029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 846/1673] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:45 2022] PUT /identity/v3/projects/c2588fe3d23f434d8d329da0e3028d74/users/6b373accd7804d19927cb077925444c3/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:45.797199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:45.896067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 828/1674] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:45 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:45.897939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:46.009921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09ebd7-2d5f-4d24-80a0-13574b1f0713 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:46.031187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c093d12-0582-4032-bd4e-bac469726b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:46.032539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093d12-0582-4032-bd4e-bac469726b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:46.033237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093d12-0582-4032-bd4e-bac469726b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:46.033644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c093d12-0582-4032-bd4e-bac469726b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:46.034481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c093d12-0582-4032-bd4e-bac469726b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:46.037006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c093d12-0582-4032-bd4e-bac469726b7f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:46.070284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 847/1675] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:46.074569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:46.105820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6364f09-8dd4-411b-9acd-6a2737133740 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:46.120048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c0389b-3a12-4431-b3bb-62801dc08f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:46.121376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0389b-3a12-4431-b3bb-62801dc08f79 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:46.121862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0389b-3a12-4431-b3bb-62801dc08f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:46.139458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c0389b-3a12-4431-b3bb-62801dc08f79 admin admin] PATH_INFO: `/v3/projects/c2588fe3d23f434d8d329da0e3028d74/users/6b373accd7804d19927cb077925444c3/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:46.147766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0389b-3a12-4431-b3bb-62801dc08f79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2588fe3d23f434d8d329da0e3028d74, user_id=6b373accd7804d19927cb077925444c3, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:46.161542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c0389b-3a12-4431-b3bb-62801dc08f79 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:46.230420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 829/1676] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:46 2022] PUT /identity/v3/projects/c2588fe3d23f434d8d329da0e3028d74/users/6b373accd7804d19927cb077925444c3/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:20:46.305662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32959e9-fe94-4344-9a90-a10e1d15c1ba None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:46.306553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32959e9-fe94-4344-9a90-a10e1d15c1ba None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:46.307181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32959e9-fe94-4344-9a90-a10e1d15c1ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:46.338453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c32959e9-fe94-4344-9a90-a10e1d15c1ba None None] MFA Rules not processed for user `ef09a3d1aed344eeb6a5100ddd420e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:46.340224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c32959e9-fe94-4344-9a90-a10e1d15c1ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:46.540912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 848/1677] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:46.826991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f72c6-5f77-4f54-aa5b-009ee58eca12 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:46.827421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f72c6-5f77-4f54-aa5b-009ee58eca12 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:46.827791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f72c6-5f77-4f54-aa5b-009ee58eca12 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:46.830612 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 830/1678] 10.222.0.13 () {54 vars in 912 bytes} [Tue Oct 4 10:20:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:46.835656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:46.836238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:46.836796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:46.885871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] MFA Rules not processed for user `0e568115c67243d3904359b84e0a790e`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:46.890785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:46.948039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee503d2-7208-4640-9812-9ce73e124e96 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:46.949228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee503d2-7208-4640-9812-9ce73e124e96 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:46.949722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee503d2-7208-4640-9812-9ce73e124e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:47.005434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1ee503d2-7208-4640-9812-9ce73e124e96 None None] MFA Rules not processed for user `7c9b4978f33247aea4d5768d98120e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:47.007870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1ee503d2-7208-4640-9812-9ce73e124e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:47.048875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 849/1679] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:47.109952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:47.122040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f5670f-997c-4f72-81ba-08156a5ce0e6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:47.127769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec78fad-f541-40c4-af9e-e30931398dbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:47.128651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78fad-f541-40c4-af9e-e30931398dbc service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:47.129086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78fad-f541-40c4-af9e-e30931398dbc service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:47.129594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec78fad-f541-40c4-af9e-e30931398dbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:47.138724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec78fad-f541-40c4-af9e-e30931398dbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:47.140351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec78fad-f541-40c4-af9e-e30931398dbc service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:47.152152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 850/1680] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:47 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:47.186268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 831/1681] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:47.196143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:47.196414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:47.196693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:47.223262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] MFA Rules not processed for user `7c9b4978f33247aea4d5768d98120e71`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:47.224847 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:47.232941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 851/1682] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:48.312152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:48.313370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:48.314453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:48.344734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:48.358759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:48.360463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:48.400833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] MFA Rules not processed for user `6b373accd7804d19927cb077925444c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:48.402327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:48.476264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:48.547375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f64e3f-12d8-48d5-849e-87f7f228ea0d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:48.571053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b07f20-98fb-4fa7-b054-69eada6bfaf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:48.571668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b07f20-98fb-4fa7-b054-69eada6bfaf1 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:48.571956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b07f20-98fb-4fa7-b054-69eada6bfaf1 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:48.572237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b07f20-98fb-4fa7-b054-69eada6bfaf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:48.607242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b07f20-98fb-4fa7-b054-69eada6bfaf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:48.609953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b07f20-98fb-4fa7-b054-69eada6bfaf1 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:48.647790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 832/1683] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:48 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 337 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:48.672376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 852/1684] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:50.924667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:50.926073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:51.061477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fd4c2-2b15-4964-9be1-948fdf997104 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.062394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fd4c2-2b15-4964-9be1-948fdf997104 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.062814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61fd4c2-2b15-4964-9be1-948fdf997104 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.065926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 853/1685] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:51.075478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbeac73-e3c0-4fbe-a0d4-068a01eb2d58 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.076034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbeac73-e3c0-4fbe-a0d4-068a01eb2d58 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.076493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbeac73-e3c0-4fbe-a0d4-068a01eb2d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.116733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9fbeac73-e3c0-4fbe-a0d4-068a01eb2d58 None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:51.117988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9fbeac73-e3c0-4fbe-a0d4-068a01eb2d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:51.140820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6398a1e3-9f78-45ab-9f82-bddf67c19a7b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:51.155113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:51.156491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.156991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.157376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] PATH_INFO: `/v3/users/201776c863f14727b3bb7b7912e89f46` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.159697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] RBAC: Authorizing `identity:delete_user(user_id=201776c863f14727b3bb7b7912e89f46)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:51.162422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:51.290241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '201776c863f14727b3bb7b7912e89f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:51.291928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '201776c863f14727b3bb7b7912e89f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:51.305345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '201776c863f14727b3bb7b7912e89f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:51.306368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '201776c863f14727b3bb7b7912e89f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:51.318407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 854/1686] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:20:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:20:51.343182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f85177f4-24fb-423b-a093-8d943bf7828f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '201776c863f14727b3bb7b7912e89f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:51.389320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061c35-b6da-4bf7-9c09-8a688d0802f0 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.389905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061c35-b6da-4bf7-9c09-8a688d0802f0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.390367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c061c35-b6da-4bf7-9c09-8a688d0802f0 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.393063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 855/1687] 10.222.0.13 () {54 vars in 961 bytes} [Tue Oct 4 10:20:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:20:51.396649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 833/1688] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:50 2022] DELETE /identity/v3/users/201776c863f14727b3bb7b7912e89f46 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:51.399126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582cc33-daca-4729-b870-b7e5a49a4fef None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.399126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582cc33-daca-4729-b870-b7e5a49a4fef None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.399920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d582cc33-daca-4729-b870-b7e5a49a4fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.401448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.401927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.402371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.458446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:51.460169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:51.472661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d582cc33-daca-4729-b870-b7e5a49a4fef None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:51.475534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d582cc33-daca-4729-b870-b7e5a49a4fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:51.491329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:51.696068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 834/1689] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:51.699381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:51.700586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:51.703978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:51.826220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 856/1690] 10.222.0.13 () {58 vars in 1058 bytes} [Tue Oct 4 10:20:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:51.829027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:51.832380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:51.914926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:51.916247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:51.997686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba7b619-f40e-4e2e-9d12-2fa1ebe0477a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:52.010875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:52.011872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:52.012452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:52.012947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:52.037107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:52.042858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:52.068101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 857/1691] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:52.070911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:52.174801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216195da-ab7e-4bf8-bfc4-0daa08fbe84f service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:52.191671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 835/1692] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:20:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:52.192550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:52.202479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:52.216138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75021a68-eaca-4e2d-af91-2181b61f3cb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:52.225319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021a68-eaca-4e2d-af91-2181b61f3cb7 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:52.225677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021a68-eaca-4e2d-af91-2181b61f3cb7 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:52.228693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75021a68-eaca-4e2d-af91-2181b61f3cb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:52.268970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3af740b-3e2c-4ed1-94b1-5f31c7cea173 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:52.283986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:52.290631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:52.291074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:52.291349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] PATH_INFO: `/v3/users/6dc8e0d5cbb347eb8cbce3c49fda452d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:52.293592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc8e0d5cbb347eb8cbce3c49fda452d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:52.298588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:52.318450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75021a68-eaca-4e2d-af91-2181b61f3cb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:52.318450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75021a68-eaca-4e2d-af91-2181b61f3cb7 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:52.386625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 858/1693] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:52.433641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc8e0d5cbb347eb8cbce3c49fda452d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:52.434332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc8e0d5cbb347eb8cbce3c49fda452d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:52.443499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc8e0d5cbb347eb8cbce3c49fda452d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:52.445135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc8e0d5cbb347eb8cbce3c49fda452d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:52.462413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc8e0d5cbb347eb8cbce3c49fda452d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:52.482931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:52.507081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 836/1694] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:52 2022] DELETE /identity/v3/users/6dc8e0d5cbb347eb8cbce3c49fda452d => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:52.510468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:52.511333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:52.512171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:52.531360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:52.533315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:52.539073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:52.639566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:52.641476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:52.725990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 859/1695] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:52.777894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:52.779606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:52.902921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7afb42-2aad-4267-8e7d-47fb53c59b58 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:52.913065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c89b68-3763-4c45-acc7-704b1ccefacb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:52.917185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:52.917846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:52.918124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:52.918441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:52.920381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:52.921307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:52.921738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:52.922356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:52.930858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:52.945111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:52.946883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:53.014098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 860/1696] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:20:53.018359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:53.019990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:53.050849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6119c7-84a7-4a1b-8657-2eef20d4f5c4 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:53.064944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:53.065762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:53.066187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:53.066617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] PATH_INFO: `/v3/users/2ea5cf0e020a4d60903767f4e693a104` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:53.072465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ea5cf0e020a4d60903767f4e693a104)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:53.074287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: WARNING py.warnings [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:20:53.074287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: warnings.warn(msg) Oct 04 10:20:53.074287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: Oct 04 10:20:53.075827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:53.139529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:53.141280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7a59ad-14d0-456b-8be8-0853ad26f135 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:53.185367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 837/1697] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:20:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:20:53.255758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ea5cf0e020a4d60903767f4e693a104'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.257125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ea5cf0e020a4d60903767f4e693a104'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.300167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ea5cf0e020a4d60903767f4e693a104'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.301559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ea5cf0e020a4d60903767f4e693a104'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.363215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ea5cf0e020a4d60903767f4e693a104'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.405411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 861/1698] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:53 2022] DELETE /identity/v3/users/2ea5cf0e020a4d60903767f4e693a104 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:20:53.411806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:53.414528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:53.416053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:53.466318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:53.467822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:53.709386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 838/1699] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:53.725841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:53.727342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:53.767324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af614af1-4f2d-403f-80a3-03c28ec5d40b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:53.781945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:53.783012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:53.783290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:53.784008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] PATH_INFO: `/v3/users/3d4a02d754dd449fb4fc81cd34620c1a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:53.789627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d4a02d754dd449fb4fc81cd34620c1a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:53.794851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:53.830776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:53.832585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:53.835191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:53.882307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:53.949156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d4a02d754dd449fb4fc81cd34620c1a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.951703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d4a02d754dd449fb4fc81cd34620c1a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.989337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d4a02d754dd449fb4fc81cd34620c1a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:53.995637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d4a02d754dd449fb4fc81cd34620c1a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:54.051205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc36e9f-f613-4e8b-956e-f14b4ee92940 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:54.061531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:54.065475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d4a02d754dd449fb4fc81cd34620c1a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:54.067443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:54.068646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:54.069901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:54.075373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:54.099798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 862/1700] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:53 2022] DELETE /identity/v3/users/3d4a02d754dd449fb4fc81cd34620c1a => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:54.123331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:54.127556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:54.159334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 839/1701] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:55.505800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:55.507967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:55.525536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:55.527054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:55.560827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900828b-cc3a-4b82-bf14-ebddac0e81a7 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:55.697573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af61c59-dc4e-4f05-a66c-5ea50e3cfc4f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:55.712512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:55.713481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:55.713923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:55.714375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] PATH_INFO: `/v3/projects/9a791fa3fed340eca53f87f8caee245e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:55.717800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a791fa3fed340eca53f87f8caee245e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:55.721025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:55.778013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:55.779231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:55.780865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:55.781382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] PATH_INFO: `/v3/projects/97519cdb8d864cf9b5f223f675fac2ac` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:55.788816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97519cdb8d864cf9b5f223f675fac2ac)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:55.790430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: WARNING py.warnings [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 04 10:20:55.790430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: warnings.warn(msg) Oct 04 10:20:55.790430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: Oct 04 10:20:55.797792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:55.834914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a791fa3fed340eca53f87f8caee245e', 'request_id': 'req-2bb6a120-0b6d-4005-adea-805b186c0ad9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:55.847158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a791fa3fed340eca53f87f8caee245e', 'request_id': 'req-2bb6a120-0b6d-4005-adea-805b186c0ad9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:55.864899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a791fa3fed340eca53f87f8caee245e', 'request_id': 'req-2bb6a120-0b6d-4005-adea-805b186c0ad9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:55.883786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] The token cache is being invalidate because project 9a791fa3fed340eca53f87f8caee245e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:20:55.884032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:55.889573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bb6a120-0b6d-4005-adea-805b186c0ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:55.892148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 863/1702] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:20:55 2022] DELETE /identity/v3/projects/9a791fa3fed340eca53f87f8caee245e => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:55.897132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58940a0-d32d-4eed-9f59-6f98972077ec None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:55.898283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58940a0-d32d-4eed-9f59-6f98972077ec None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:55.898283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58940a0-d32d-4eed-9f59-6f98972077ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:55.945325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e58940a0-d32d-4eed-9f59-6f98972077ec None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:55.947148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e58940a0-d32d-4eed-9f59-6f98972077ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:55.967473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:55.980586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97519cdb8d864cf9b5f223f675fac2ac', 'request_id': 'req-eed308df-f221-415b-a2b0-630f076296b9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.034642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97519cdb8d864cf9b5f223f675fac2ac', 'request_id': 'req-eed308df-f221-415b-a2b0-630f076296b9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.072881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97519cdb8d864cf9b5f223f675fac2ac', 'request_id': 'req-eed308df-f221-415b-a2b0-630f076296b9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.126821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 864/1703] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:20:56.133404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:56.133841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:56.134281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:56.159639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] The token cache is being invalidate because project 97519cdb8d864cf9b5f223f675fac2ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:20:56.159937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.160749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.168238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 840/1704] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:20:55 2022] DELETE /identity/v3/projects/97519cdb8d864cf9b5f223f675fac2ac => generated 0 bytes in 640 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:20:56.170352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:56.174624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:56.174843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:56.176764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:56.203925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 865/1705] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:56.210486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:56.211930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed308df-f221-415b-a2b0-630f076296b9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:56.217105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:56.220711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:56.220711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:56.220711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] PATH_INFO: `/v3/users/6b373accd7804d19927cb077925444c3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:56.223016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b373accd7804d19927cb077925444c3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:56.223752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:56.242838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27c79fa-37db-4d6a-97a0-0e530df4539b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:56.256110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:56.257439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:56.257439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:56.257439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:56.260672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:56.260672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:56.260961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c05478-32c9-4a14-8af7-43c292aacdda admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:56.289270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 866/1706] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:56.293555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:56.294956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:56.296066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:56.375211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] MFA Rules not processed for user `0e568115c67243d3904359b84e0a790e`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:56.375611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:56.412567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:56.419491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b373accd7804d19927cb077925444c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.419491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b373accd7804d19927cb077925444c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.452786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b373accd7804d19927cb077925444c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.458078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b373accd7804d19927cb077925444c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.530986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b373accd7804d19927cb077925444c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.576863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 841/1707] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:56 2022] DELETE /identity/v3/users/6b373accd7804d19927cb077925444c3 => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:56.580592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:56.584809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:56.613765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 867/1708] 10.222.0.36 () {58 vars in 1015 bytes} [Tue Oct 4 10:20:56 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:56.623728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:56.638007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:56.685584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a6f015-e7e3-4abb-8ccb-9f1831344766 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:56.771784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcc4254-0184-4a5d-aca1-1cef1f528dfb admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:56.801594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:56.806172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:56.808943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:56.809901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] PATH_INFO: `/v3/projects/6c286d36379c4275b5d533acd2431502` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:56.816840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c286d36379c4275b5d533acd2431502)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:56.819082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:56.936965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c286d36379c4275b5d533acd2431502', 'request_id': 'req-6410ac09-f63e-49b6-b807-d06d37785921'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.952121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c286d36379c4275b5d533acd2431502', 'request_id': 'req-6410ac09-f63e-49b6-b807-d06d37785921'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:56.969344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c286d36379c4275b5d533acd2431502', 'request_id': 'req-6410ac09-f63e-49b6-b807-d06d37785921'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.034294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:57.036291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] The token cache is being invalidate because project 6c286d36379c4275b5d533acd2431502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:20:57.039499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:57.039938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.041858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:57.042470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.043834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:57.049726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:57.058112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 842/1709] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:20:56 2022] DELETE /identity/v3/projects/6c286d36379c4275b5d533acd2431502 => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:57.059351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:57.061019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:57.064374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:57.223886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 868/1710] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:56 2022] POST /identity/v3/projects => generated 357 bytes in 597 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:57.223886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:57.252069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:57.264959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410ac09-f63e-49b6-b807-d06d37785921 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:57.283777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:57.407692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:57.408335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:57.408623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:57.408893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:57.418707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:57.459536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:57.460991 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:57.478145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 843/1711] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:20:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:20:57.480300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:57.483601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:57.534280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:57.698039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c00e6-9d5f-48cc-8612-0f359ecd3c3d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:57.702573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35faa04-69fb-4aa3-bfbf-4eb51dc98690 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:57.714856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:57.715873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:57.716367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:57.716793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] PATH_INFO: `/v3/users/2abcbc861c59499ca8758f887de7830e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:57.718162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:57.719102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:57.719382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:57.719549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2abcbc861c59499ca8758f887de7830e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:57.719644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] PATH_INFO: `/v3/projects/4bcab3ae3d924513aa07f4feea89b2ce` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:57.722394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:57.734715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bcab3ae3d924513aa07f4feea89b2ce)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:57.750293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:57.852438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2abcbc861c59499ca8758f887de7830e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.854397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2abcbc861c59499ca8758f887de7830e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.879124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2abcbc861c59499ca8758f887de7830e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.880132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2abcbc861c59499ca8758f887de7830e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.917519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2abcbc861c59499ca8758f887de7830e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.924921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bcab3ae3d924513aa07f4feea89b2ce', 'request_id': 'req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.936843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bcab3ae3d924513aa07f4feea89b2ce', 'request_id': 'req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.945623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bcab3ae3d924513aa07f4feea89b2ce', 'request_id': 'req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.946647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 844/1712] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:20:57 2022] DELETE /identity/v3/users/2abcbc861c59499ca8758f887de7830e => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:57.950002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:57.955430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:57.992039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] The token cache is being invalidate because project 4bcab3ae3d924513aa07f4feea89b2ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:20:57.994154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:57.996799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3f960fc0-fe15-4bd3-99b2-1e24bdab4cf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:58.008759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 869/1713] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:20:57 2022] DELETE /identity/v3/projects/4bcab3ae3d924513aa07f4feea89b2ce => generated 0 bytes in 787 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:58.014034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023fe9-a191-40db-a384-1f451f037ca2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.014638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023fe9-a191-40db-a384-1f451f037ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.015789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91023fe9-a191-40db-a384-1f451f037ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.066702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:58.091510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-91023fe9-a191-40db-a384-1f451f037ca2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:58.099914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-91023fe9-a191-40db-a384-1f451f037ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:58.152934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5c9f4-8a37-44a4-bc25-949275352a7c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.162092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.163183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.163680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.164169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.165029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.167262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.244108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 845/1714] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:57 2022] POST /identity/v3/users => generated 279 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:58.250842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.255622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:58.256127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 870/1715] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:58.264957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.265915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.267848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.297342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cc0cff-b260-424f-9afb-1acb5f87395e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.320216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.320946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.323375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.323727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.324434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.330745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:58.331074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.339744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:58.372128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 871/1716] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:58.373432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.396957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 846/1717] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:58.397796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cb1553-7cbd-4512-8790-6622d3f72227 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.398078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.417612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.418279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.418567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.418860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.419513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:58.419819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.432660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.432937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85229f7b-d011-4779-b0b9-595038ee3d2c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.441643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.442274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.442700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.445061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.446335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.448072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.483088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 847/1718] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:58.490952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 872/1719] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:20:58.498651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.503247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.523324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72852e03-23ce-42d6-9230-f51f41128289 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.526325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424ccea9-fbe7-4c1c-9df2-ab52bc38b66a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.544594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79562ab7-e3be-433a-a87a-154f49c42977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.545501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.545744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79562ab7-e3be-433a-a87a-154f49c42977 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.546117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.546410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79562ab7-e3be-433a-a87a-154f49c42977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.546509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.546689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.547281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79562ab7-e3be-433a-a87a-154f49c42977 admin admin] PATH_INFO: `/v3/projects/369bc7ac4b1c4b58afc1b1addf818797/users/fc211049c85d4072b0d725074c4c457f/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.547525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.550045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.596584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79562ab7-e3be-433a-a87a-154f49c42977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=369bc7ac4b1c4b58afc1b1addf818797, user_id=fc211049c85d4072b0d725074c4c457f, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.602242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79562ab7-e3be-433a-a87a-154f49c42977 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.629575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 848/1720] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:58 2022] PUT /identity/v3/projects/369bc7ac4b1c4b58afc1b1addf818797/users/fc211049c85d4072b0d725074c4c457f/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:58.641299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.642576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.643196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.666560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 873/1721] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:58 2022] POST /identity/v3/projects => generated 403 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:58.667496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.717947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] MFA Rules not processed for user `fc211049c85d4072b0d725074c4c457f`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:58.720601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:58.767728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:58.891196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87be25ef-beb9-4510-90c3-e1bdab3dc49b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.903350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.903976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.904252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.904602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.905375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.915652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:58.919190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 849/1722] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:58 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:58.919943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.923365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:58.960408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1a9256-de10-4b6a-a40d-f3677147d4fe None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:58.967971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 874/1723] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:58 2022] POST /identity/v3/users => generated 302 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:58.972545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:58.976592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:58.977462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:58.977946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:58.978097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] PATH_INFO: `/v3/projects/c2588fe3d23f434d8d329da0e3028d74` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:58.983251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2588fe3d23f434d8d329da0e3028d74)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:58.989318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.002223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf247b-55de-4564-8c90-64c840997c39 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.024112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.024766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.025027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.025295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.028221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.030263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.110575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 875/1724] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:58 2022] POST /identity/v3/projects => generated 353 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:59.119746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.170932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2588fe3d23f434d8d329da0e3028d74', 'request_id': 'req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:59.207042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2588fe3d23f434d8d329da0e3028d74', 'request_id': 'req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:59.209006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:20:59.225152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2588fe3d23f434d8d329da0e3028d74', 'request_id': 'req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:59.259931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] The token cache is being invalidate because project c2588fe3d23f434d8d329da0e3028d74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:20:59.259931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:59.267396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:20:59.275885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 850/1725] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:20:58 2022] DELETE /identity/v3/projects/c2588fe3d23f434d8d329da0e3028d74 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:59.276522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.281161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:59.314791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe7d17a-e408-4af8-98e7-6fff8e57bbd2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.333225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380fdb3-b749-4dcf-b442-ab40cd0273d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.334055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fdb3-b749-4dcf-b442-ab40cd0273d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.334515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fdb3-b749-4dcf-b442-ab40cd0273d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.334949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380fdb3-b749-4dcf-b442-ab40cd0273d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.335771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fdb3-b749-4dcf-b442-ab40cd0273d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.340363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380fdb3-b749-4dcf-b442-ab40cd0273d1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.412632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 876/1726] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:59.417660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.417964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.418261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.511849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca276bcb-c541-4854-b0dc-7ddbbe867ca4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.517874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699851f1-836c-4f7f-bc99-ac58ec81f31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.518575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699851f1-836c-4f7f-bc99-ac58ec81f31d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.518855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699851f1-836c-4f7f-bc99-ac58ec81f31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.519125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699851f1-836c-4f7f-bc99-ac58ec81f31d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.519761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699851f1-836c-4f7f-bc99-ac58ec81f31d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.521012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:59.521885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699851f1-836c-4f7f-bc99-ac58ec81f31d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.523250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:59.559177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 877/1727] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:59.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.575099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 851/1728] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:20:59 2022] POST /identity/v3/users => generated 283 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:20:59.577877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.578095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:59.578708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.579215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.611889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6b5c8-d8bc-472e-8e3d-862918eec4c1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.640539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.642640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.645463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:20:59.645697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.645986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.648024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:20:59.649143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.651911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.670094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 852/1729] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:20:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:20:59.671569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.678120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 878/1730] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:59.680615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.694644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cea904-12b1-40a8-a025-1c9490536b0c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.696341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9198cb95-09a4-41ce-8879-f5f5686132c3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.708736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.709519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.709805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.710235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.713169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.713786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.714819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.715486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.716134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.716562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.717341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:20:59.717762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.721739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.756854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 879/1731] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:20:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:20:59.757775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.758810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 853/1732] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:20:59.762720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.785736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97587dc7-551d-491a-9b89-c86c4ec7477c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.800476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.801178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.801454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.801723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] PATH_INFO: `/v3/projects/c0261626d1a241ecaa9478dacebddb75/users/57e4399f676447ab9d82e8111b015264/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.822097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b636ffa8-ee97-4530-90e6-9390a047fc52 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:20:59.843266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0261626d1a241ecaa9478dacebddb75, user_id=57e4399f676447ab9d82e8111b015264, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.845979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:20:59.846415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.849098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:20:59.850455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:20:59.851596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] PATH_INFO: `/v3/projects/30c0356e91cd493d89d93c5b909ba44c/users/6139bd5a21074118a67a9ce999b44f82/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:20:59.871378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 854/1733] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:59 2022] PUT /identity/v3/projects/c0261626d1a241ecaa9478dacebddb75/users/57e4399f676447ab9d82e8111b015264/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:59.872271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:20:59.903636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=30c0356e91cd493d89d93c5b909ba44c, user_id=6139bd5a21074118a67a9ce999b44f82, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:20:59.916759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:20:59.997929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 880/1734] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:20:59 2022] PUT /identity/v3/projects/30c0356e91cd493d89d93c5b909ba44c/users/6139bd5a21074118a67a9ce999b44f82/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:20:59.997929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:00.088648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238d9a1a-10d8-4785-b7a5-3545070d4ff7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:00.144405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:00.284011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833ce78d-ef57-4409-932b-c383a7eaf5bd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:00.298707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:00.304915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dde3dcf-4028-43fc-a6b3-aad45d7f9477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:00.315276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde3dcf-4028-43fc-a6b3-aad45d7f9477 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.315489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde3dcf-4028-43fc-a6b3-aad45d7f9477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.315646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dde3dcf-4028-43fc-a6b3-aad45d7f9477 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.316381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde3dcf-4028-43fc-a6b3-aad45d7f9477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:00.326098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dde3dcf-4028-43fc-a6b3-aad45d7f9477 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:00.363419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 881/1735] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:20:59 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:00.363419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.363419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.363419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.367056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:00.367594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.367838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.368185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.371019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:00.372957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:00.425425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] MFA Rules not processed for user `6139bd5a21074118a67a9ce999b44f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:00.428543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:00.435041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 855/1736] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:20:59 2022] POST /identity/v3/projects => generated 393 bytes in 563 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:00.436640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:00.529770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663982ab-c7a5-4de8-be6f-b631423e8d55 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:00.540276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:00.540962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.541244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.541623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] PATH_INFO: `/v3/projects/c0261626d1a241ecaa9478dacebddb75/users/57e4399f676447ab9d82e8111b015264/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.548230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0261626d1a241ecaa9478dacebddb75, user_id=57e4399f676447ab9d82e8111b015264, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:00.554452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:00.580044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 856/1737] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:00 2022] PUT /identity/v3/projects/c0261626d1a241ecaa9478dacebddb75/users/57e4399f676447ab9d82e8111b015264/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:00.583188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:00.632776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 882/1738] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:00.633958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:00.640513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:00.675204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb5ebc-4e53-49fd-8dcf-e0c1d243e000 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:00.820055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2203907-7e66-49cf-b203-7fe340e433ac admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:00.827206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c9a87-e875-4bc3-8e73-043631794863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:00.828371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c9a87-e875-4bc3-8e73-043631794863 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.828883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c9a87-e875-4bc3-8e73-043631794863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.830189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c9a87-e875-4bc3-8e73-043631794863 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.831460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c9a87-e875-4bc3-8e73-043631794863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:00.833871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c9a87-e875-4bc3-8e73-043631794863 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:00.864803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:00.865685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.866081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.866409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] PATH_INFO: `/v3/projects/06d7abc3014648a38ceffeffd1d4f51d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.868733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06d7abc3014648a38ceffeffd1d4f51d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:00.874552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:00.874818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 857/1739] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:00 2022] POST /identity/v3/users => generated 297 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:00.878190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:00.878775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:00.879232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:00.960745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] MFA Rules not processed for user `57e4399f676447ab9d82e8111b015264`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:00.962345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:01.010796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06d7abc3014648a38ceffeffd1d4f51d', 'request_id': 'req-f6b14890-2cb7-4741-954a-5b0d1ca9f798'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:01.027491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06d7abc3014648a38ceffeffd1d4f51d', 'request_id': 'req-f6b14890-2cb7-4741-954a-5b0d1ca9f798'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:01.039310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:01.043130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06d7abc3014648a38ceffeffd1d4f51d', 'request_id': 'req-f6b14890-2cb7-4741-954a-5b0d1ca9f798'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:01.067743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] The token cache is being invalidate because project 06d7abc3014648a38ceffeffd1d4f51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:01.068063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:01.074292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f6b14890-2cb7-4741-954a-5b0d1ca9f798 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:01.076330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 883/1740] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:00 2022] DELETE /identity/v3/projects/06d7abc3014648a38ceffeffd1d4f51d => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:01.077682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21f001-a3f8-43c5-a549-422e6763dd50 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.077968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21f001-a3f8-43c5-a549-422e6763dd50 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.078259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a21f001-a3f8-43c5-a549-422e6763dd50 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.080224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 884/1741] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:21:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:21:01.081164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21f001-a3f8-43c5-a549-422e6763dd50 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:01.083652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a21f001-a3f8-43c5-a549-422e6763dd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:01.120348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:01.202982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 858/1742] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:00 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:01.205993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:01.288947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:01.315650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a21f001-a3f8-43c5-a549-422e6763dd50 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:01.328088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ab9670-7f86-4dac-87dc-8fb1b1da754e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:01.329482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9670-7f86-4dac-87dc-8fb1b1da754e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.329606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9670-7f86-4dac-87dc-8fb1b1da754e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.329960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ab9670-7f86-4dac-87dc-8fb1b1da754e admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.332875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab9670-7f86-4dac-87dc-8fb1b1da754e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:01.332875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ab9670-7f86-4dac-87dc-8fb1b1da754e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:01.339138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a79c24b-26e5-4a42-9932-63347078c3c1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:01.359897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:01.363013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.365876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.368062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.371576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:01.375012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:01.398899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 885/1743] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:01 2022] POST /identity/v3/projects => generated 405 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:01.402945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.402945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.403187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.415959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 859/1744] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:01 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:01.418985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:01.424400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:01.447201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:01.465944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:01.647081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3855a-539a-4d4b-8954-8f7c7f4959ac admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:01.727185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 886/1745] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:01.728079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:01.753361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5924cd-1b84-4ca6-893d-6731d76cc93e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:01.762237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ecb45-17f8-4b2f-824f-8baa342f3828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:01.763175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ecb45-17f8-4b2f-824f-8baa342f3828 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.763467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ecb45-17f8-4b2f-824f-8baa342f3828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.763740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ecb45-17f8-4b2f-824f-8baa342f3828 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.764386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ecb45-17f8-4b2f-824f-8baa342f3828 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:01.767762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ecb45-17f8-4b2f-824f-8baa342f3828 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:01.800390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:01.801280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.801721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.802359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.825727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 887/1746] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:01 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:01.832424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b316c1c-cf3d-461c-bed4-80985c670c36 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:01.832742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b316c1c-cf3d-461c-bed4-80985c670c36 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:01.833021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b316c1c-cf3d-461c-bed4-80985c670c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:01.882096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3b316c1c-cf3d-461c-bed4-80985c670c36 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:01.884996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b316c1c-cf3d-461c-bed4-80985c670c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:01.950419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:01.951990 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.031013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:02.036625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 860/1747] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:01 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:02.056969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.073729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 888/1748] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:21:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:02.077932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.078418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.078651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.127867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9765d274-523b-40ff-a752-24bb2800849d service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.134355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:02.135868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:02.154817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.155849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.156632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.156923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.157679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.165673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 889/1749] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:02.165772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.165851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.187795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 861/1750] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:02.189592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.191877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab37fa5-b090-4662-ad3f-b5ba9716a6bb None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.216146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cfe6de-e959-4051-b7d7-7e004b33c8d8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.221674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.222331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.222640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.222880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.223533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.225233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.243562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.244084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.244359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.244633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.245283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:02.245580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.247564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.264931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 890/1751] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:02 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:02.266115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.269838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 862/1752] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:02.271050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.284301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962eef51-c936-4171-af34-38fe73597472 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.301379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.302074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.303619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.303982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.306787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b26df7-5561-4503-b7b1-f208ee773f93 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.312619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:02.324804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:02.326805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.327649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.327875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.328138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] PATH_INFO: `/v3/projects/b2e4d8dddc214867b6e98a31f1dabf4f/users/a49b1016e26f41b78a6073e8c0371568/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.367542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2e4d8dddc214867b6e98a31f1dabf4f, user_id=a49b1016e26f41b78a6073e8c0371568, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.370439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.410235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 891/1753] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:02 2022] PUT /identity/v3/projects/b2e4d8dddc214867b6e98a31f1dabf4f/users/a49b1016e26f41b78a6073e8c0371568/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:02.416863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.473712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.477273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.515518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 863/1754] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:02 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:02.518330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.545399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c8c27c-d745-4d16-976b-f94e2cfd1f4d service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.575453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966c75aa-151e-4d48-a8eb-608b07c6dcd4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:02.590719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.591378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.591659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.591928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] PATH_INFO: `/v3/projects/51b88dfa0e7a4c4293833d9e86a84072/users/ff1e631da63547e6b40dc7511f82f85d/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.648451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b88dfa0e7a4c4293833d9e86a84072, user_id=ff1e631da63547e6b40dc7511f82f85d, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.667722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.687567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1676d1-0891-4a04-8ca8-463bd6d66fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:02.691379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1676d1-0891-4a04-8ca8-463bd6d66fac admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.692982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1676d1-0891-4a04-8ca8-463bd6d66fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.693300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1676d1-0891-4a04-8ca8-463bd6d66fac admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.696313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1676d1-0891-4a04-8ca8-463bd6d66fac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:02.703677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1676d1-0891-4a04-8ca8-463bd6d66fac admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:02.733961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 892/1755] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:02 2022] PUT /identity/v3/projects/51b88dfa0e7a4c4293833d9e86a84072/users/ff1e631da63547e6b40dc7511f82f85d/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:02.735905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:02.743639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:02.780422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 864/1756] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:02 2022] POST /identity/v3/projects => generated 363 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:02.780422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:02.780677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:02.781047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:02.860847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] MFA Rules not processed for user `a49b1016e26f41b78a6073e8c0371568`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:02.862077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:03.052121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e099f-517b-4c16-aa9f-a7873776ee77 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.091301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 865/1757] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:03.092186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:03.118735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:03.145144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:03.237181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.241300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.241590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.241872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.260004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1dc593-02e1-43f0-bea8-cd434e5ee2d8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.262788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.264399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:03.268256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.278019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.278520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.279034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.280113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.283001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:03.292885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 893/1758] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:03.293892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:03.305779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 866/1759] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:03.306735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:03.320058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ede4c9-8b70-45c4-af9d-7c9249e018d7 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.323641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a27619-d854-47ff-a9ba-9e60c7d8778b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.328907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.329576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.330124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.330491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.331431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.333022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:03.334795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.335715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.336148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.336560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.337528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.347612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:03.390109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 867/1760] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:03 2022] POST /identity/v3/projects => generated 391 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:03.391057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:03.409529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 894/1761] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:03 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:03.413448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:03.446449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ff8d07-4536-4c29-9f1b-8a6141b09bf4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.644052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1d3b5-24c8-44c2-aa5e-a59fca105234 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.666647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.667731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.668187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.668606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] PATH_INFO: `/v3/projects/51b88dfa0e7a4c4293833d9e86a84072/users/ff1e631da63547e6b40dc7511f82f85d/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.677749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] RBAC: Authorizing `identity:create_grant(project_id=51b88dfa0e7a4c4293833d9e86a84072, user_id=ff1e631da63547e6b40dc7511f82f85d, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.693790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:03.728657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2179a6c-507a-47aa-834d-fe662196af7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.729699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2179a6c-507a-47aa-834d-fe662196af7d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.729863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2179a6c-507a-47aa-834d-fe662196af7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.731209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2179a6c-507a-47aa-834d-fe662196af7d admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.732036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2179a6c-507a-47aa-834d-fe662196af7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.739018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2179a6c-507a-47aa-834d-fe662196af7d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:03.746653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 868/1762] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:03 2022] PUT /identity/v3/projects/51b88dfa0e7a4c4293833d9e86a84072/users/ff1e631da63547e6b40dc7511f82f85d/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:03.748522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:03.794188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 895/1763] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:03 2022] POST /identity/v3/users => generated 302 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:03.798799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.798991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.799256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.925762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] MFA Rules not processed for user `ff1e631da63547e6b40dc7511f82f85d`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:03.952336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:03.961101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ec28a6-3126-4a6a-88a9-e9da73c9157e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:03.983422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:03.984423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:03.984876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:03.985918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:03.987048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:03.993311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.039285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 869/1764] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:04.040876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.088612 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb92a2fe-ba68-4261-b941-ff464b8430b7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.112434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.113237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.113334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.113585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.114251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.115855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.153856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 896/1765] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:03 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:04.155022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.171203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 870/1766] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:04.172506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.194972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed79aa07-d62d-4c25-9dc7-6e52d99735de None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.208792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.209049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86839bbd-2451-4b87-95ad-645c3776f9a8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.209440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.209748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.210036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.213026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.217033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.224386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.225324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.225703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.225989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] PATH_INFO: `/v3/projects/6a6a44acc6b4442aa16e706ad0242548/users/1c821aff04834c4293450d6cbdc4789a/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.240100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:04.254480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:04.263769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 897/1767] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:04.263769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.265756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6a44acc6b4442aa16e706ad0242548, user_id=1c821aff04834c4293450d6cbdc4789a, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.272927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.275951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:04.322052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 871/1768] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:04 2022] PUT /identity/v3/projects/6a6a44acc6b4442aa16e706ad0242548/users/1c821aff04834c4293450d6cbdc4789a/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:04.325306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.443333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a19f09b-0573-4e7f-bfc6-6936d57c8de6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.546491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e411679-540f-41ec-a87f-1b932e5c5495 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.563725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.564624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.564903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.565173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] PATH_INFO: `/v3/projects/0572b836ca474904991e034ee199c1f3/users/29dc579ad4cd4417b843f6c96c505d0a/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.595803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece36a2d-dd5b-4dbc-86cb-8e4ae1852a0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.596474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece36a2d-dd5b-4dbc-86cb-8e4ae1852a0f service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.596750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece36a2d-dd5b-4dbc-86cb-8e4ae1852a0f service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.597020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece36a2d-dd5b-4dbc-86cb-8e4ae1852a0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.603269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0572b836ca474904991e034ee199c1f3, user_id=29dc579ad4cd4417b843f6c96c505d0a, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.606361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.662260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 872/1769] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:04 2022] PUT /identity/v3/projects/0572b836ca474904991e034ee199c1f3/users/29dc579ad4cd4417b843f6c96c505d0a/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:04.663119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.752875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece36a2d-dd5b-4dbc-86cb-8e4ae1852a0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.754610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece36a2d-dd5b-4dbc-86cb-8e4ae1852a0f service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.782827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 898/1770] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:04 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:04.801086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.801605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.802046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.837201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de80193-d3b7-47bf-a965-1c39dc2e271e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.850057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.850808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.851187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.851623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.852486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.855113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.874978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 873/1771] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:04.879708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:04.879866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] MFA Rules not processed for user `29dc579ad4cd4417b843f6c96c505d0a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:04.881167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:04.891097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df145b-7cd8-4430-a82d-2b563cdcc980 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:04.898829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36069d89-8e87-4b76-8b8b-8082372ae418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:04.900013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069d89-8e87-4b76-8b8b-8082372ae418 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.900634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069d89-8e87-4b76-8b8b-8082372ae418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.901107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36069d89-8e87-4b76-8b8b-8082372ae418 admin admin] PATH_INFO: `/v3/projects/6a6a44acc6b4442aa16e706ad0242548/users/1c821aff04834c4293450d6cbdc4789a/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:04.905013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36069d89-8e87-4b76-8b8b-8082372ae418 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a6a44acc6b4442aa16e706ad0242548, user_id=1c821aff04834c4293450d6cbdc4789a, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:04.913274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36069d89-8e87-4b76-8b8b-8082372ae418 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:04.932445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 874/1772] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:04 2022] PUT /identity/v3/projects/6a6a44acc6b4442aa16e706ad0242548/users/1c821aff04834c4293450d6cbdc4789a/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:04.944863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:04.945368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:04.945952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.003772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] MFA Rules not processed for user `1c821aff04834c4293450d6cbdc4789a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:05.013349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.013539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:05.036669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 899/1773] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:05.044020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.073544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9067839a-dc71-4aa2-ac49-7eac71d0b7b0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.116670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:05.185648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 875/1774] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:04 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:05.186577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.229456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.230830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.231346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.232185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.233054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.235971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.300433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 900/1775] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:05 2022] POST /identity/v3/projects => generated 361 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:05.304509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.386825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeda042-babb-4ad3-a457-119cdad567ea admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.396630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.397614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.397934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.398186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1eb0c8-183f-44f2-aec9-dc118186f45c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.398490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.482387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.483134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.483509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.483885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] PATH_INFO: `/v3/users/6139bd5a21074118a67a9ce999b44f82` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.485629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6139bd5a21074118a67a9ce999b44f82)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.487638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.501852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.503442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.522349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 901/1776] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:05 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:05.524035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.552190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6572f3f-d60e-4bf7-b83a-605ebf53c3bf service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.561451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6139bd5a21074118a67a9ce999b44f82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:05.563828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6139bd5a21074118a67a9ce999b44f82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:05.566274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e2916-9fec-40a4-9708-f2b884e7cd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.566583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e2916-9fec-40a4-9708-f2b884e7cd0b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.566951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e2916-9fec-40a4-9708-f2b884e7cd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.567173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e2916-9fec-40a4-9708-f2b884e7cd0b admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.568699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e2916-9fec-40a4-9708-f2b884e7cd0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.575358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e2916-9fec-40a4-9708-f2b884e7cd0b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.585291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6139bd5a21074118a67a9ce999b44f82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:05.586600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6139bd5a21074118a67a9ce999b44f82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:05.606964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6139bd5a21074118a67a9ce999b44f82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:05.613150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 902/1777] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:05 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:05.614760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91519a2c-0325-42e7-a676-7f0e43cf84e1 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.615352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91519a2c-0325-42e7-a676-7f0e43cf84e1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.615706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91519a2c-0325-42e7-a676-7f0e43cf84e1 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.618326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 903/1778] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:21:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:21:05.619434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91519a2c-0325-42e7-a676-7f0e43cf84e1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.621914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-91519a2c-0325-42e7-a676-7f0e43cf84e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.638866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91519a2c-0325-42e7-a676-7f0e43cf84e1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.640118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 876/1779] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:05 2022] DELETE /identity/v3/users/6139bd5a21074118a67a9ce999b44f82 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:05.641098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.643332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.650525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:05.656783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:05.701870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.702528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.702808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.703146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.706442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.727624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a071aa-81d3-41c6-92d0-f88a3b1fa772 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.732793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c55808-6f96-44bf-8b2c-fdf92adcd2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.733483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c55808-6f96-44bf-8b2c-fdf92adcd2d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.733898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c55808-6f96-44bf-8b2c-fdf92adcd2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.734215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c55808-6f96-44bf-8b2c-fdf92adcd2d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.734939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c55808-6f96-44bf-8b2c-fdf92adcd2d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.737613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c55808-6f96-44bf-8b2c-fdf92adcd2d8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.752650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 877/1780] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:05.752650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efa17-5e37-442a-bfea-0649bab505fa None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.752650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efa17-5e37-442a-bfea-0649bab505fa None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.752650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24efa17-5e37-442a-bfea-0649bab505fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.772048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f24efa17-5e37-442a-bfea-0649bab505fa None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:05.773357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f24efa17-5e37-442a-bfea-0649bab505fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.781377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 878/1781] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:05.782713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.783108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.783457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.788197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.790968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.804144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 904/1782] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:05 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:05.805024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.815210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:05.816452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.819577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7aa12b-a00f-458d-911d-561c85b0f5b2 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.826172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.826948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.827300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.827710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] PATH_INFO: `/v3/projects/cb6b60f6fb264d20a50f52f99c99d210/users/a058c94040a94880bb31cdb088872073/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:05.867630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b60f6fb264d20a50f52f99c99d210, user_id=a058c94040a94880bb31cdb088872073, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:05.872435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:05.895178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 905/1783] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:05 2022] PUT /identity/v3/projects/cb6b60f6fb264d20a50f52f99c99d210/users/a058c94040a94880bb31cdb088872073/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:05.896139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.898170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:05.922402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 879/1784] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:21:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:05.926611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:05.939530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae3963-37e2-4859-8a7a-6d8e0c8baf31 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.979022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecf0c2-da9d-46eb-be02-561bab111f1b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:05.984155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:05.984896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:05.985323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:05.985753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.028846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:06.029654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.029940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.030329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.030891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.032806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.046529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 880/1785] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:06.051696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:06.054784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:06.072335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:06.088494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.088494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.094062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:06.101663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 906/1786] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:06.102640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:06.112917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8329c4f-3e38-415f-9ac4-26fb2a227d99 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:06.119138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a543571-8caf-4d7c-9629-f5a39e997960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:06.120439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a543571-8caf-4d7c-9629-f5a39e997960 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.120624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a543571-8caf-4d7c-9629-f5a39e997960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.120978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a543571-8caf-4d7c-9629-f5a39e997960 admin admin] PATH_INFO: `/v3/projects/cb6b60f6fb264d20a50f52f99c99d210/users/a058c94040a94880bb31cdb088872073/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.124348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a543571-8caf-4d7c-9629-f5a39e997960 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb6b60f6fb264d20a50f52f99c99d210, user_id=a058c94040a94880bb31cdb088872073, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.127715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a543571-8caf-4d7c-9629-f5a39e997960 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.152684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 907/1787] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:06 2022] PUT /identity/v3/projects/cb6b60f6fb264d20a50f52f99c99d210/users/a058c94040a94880bb31cdb088872073/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:06.158866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.159210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.159494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.214468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ae044a-2379-4533-92e5-25d64825caee admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:06.219518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] MFA Rules not processed for user `a058c94040a94880bb31cdb088872073`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:06.220982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:06.407868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:06.410248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.412269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.412649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.418507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 908/1788] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:06 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:06.425451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.427220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.432200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:06.458739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 881/1789] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:06.517679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51671597-5b23-46d3-b320-36d383d636d7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:06.521819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:06.522544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.522957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.523302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.553655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.561772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.575014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:06.577525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:06.589291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 909/1790] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:06.660246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2b0f9b-2b60-4f03-b850-cb0c02b7a480 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:06.681718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:06.682671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.683518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.685317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] PATH_INFO: `/v3/projects/30c0356e91cd493d89d93c5b909ba44c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.687961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30c0356e91cd493d89d93c5b909ba44c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.691833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.745169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:06.754251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd67e32-8e05-43f8-96a5-0643bfcc5f7d service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:06.760002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:06.760679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:06.760955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:06.761235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:06.763207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:06.790049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30c0356e91cd493d89d93c5b909ba44c', 'request_id': 'req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:06.792135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:06.801881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30c0356e91cd493d89d93c5b909ba44c', 'request_id': 'req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:06.817031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30c0356e91cd493d89d93c5b909ba44c', 'request_id': 'req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:06.852845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:06.854528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:06.876160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] The token cache is being invalidate because project 30c0356e91cd493d89d93c5b909ba44c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:06.876498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:06.877362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:06.883825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 882/1791] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:06 2022] DELETE /identity/v3/projects/30c0356e91cd493d89d93c5b909ba44c => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:06.886511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 910/1792] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:06 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:06.887376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:06.888838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:06.997479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e158c-94b0-44ab-9c6d-f0f6b8d743c4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:07.011060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:07.011752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:07.012034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:07.012310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] PATH_INFO: `/v3/users/ff1e631da63547e6b40dc7511f82f85d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:07.013596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff1e631da63547e6b40dc7511f82f85d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:07.016517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:07.122848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff1e631da63547e6b40dc7511f82f85d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.125150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff1e631da63547e6b40dc7511f82f85d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.138028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff1e631da63547e6b40dc7511f82f85d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.138907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff1e631da63547e6b40dc7511f82f85d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.155319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3ea0c9b3-9fe8-4e09-9aaa-38b54bcbb74a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff1e631da63547e6b40dc7511f82f85d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.177455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 883/1793] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:06 2022] DELETE /identity/v3/users/ff1e631da63547e6b40dc7511f82f85d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:07.514019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:07.538420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e79256f-b15e-458f-aa63-671b4498b998 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:07.551873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:07.702294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:07.705068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:07.707302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:07.709281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] PATH_INFO: `/v3/projects/51b88dfa0e7a4c4293833d9e86a84072` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:07.715344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51b88dfa0e7a4c4293833d9e86a84072)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:07.725279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:07.847783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51b88dfa0e7a4c4293833d9e86a84072', 'request_id': 'req-9760b289-c895-4a4b-85fe-977a9d33a4e0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.857972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51b88dfa0e7a4c4293833d9e86a84072', 'request_id': 'req-9760b289-c895-4a4b-85fe-977a9d33a4e0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.885221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51b88dfa0e7a4c4293833d9e86a84072', 'request_id': 'req-9760b289-c895-4a4b-85fe-977a9d33a4e0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.907191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] The token cache is being invalidate because project 51b88dfa0e7a4c4293833d9e86a84072 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:07.907948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.910481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9760b289-c895-4a4b-85fe-977a9d33a4e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:07.912994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 911/1794] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:07 2022] DELETE /identity/v3/projects/51b88dfa0e7a4c4293833d9e86a84072 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:07.917751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:07.918253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:07.918354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:07.960646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:07.965495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:08.048739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:08.067460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 884/1795] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:08.069157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.069452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.069731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.096918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:08.098339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:08.117945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 912/1796] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:08.118090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.135835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3427534-51aa-4889-bd0b-b39fe240be2c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.147783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.148685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.148889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.149120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.149776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:08.150081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.151889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.172924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 885/1797] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:21:08.177426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.198445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d12fd6-7e55-4a58-bb3d-a2ec23d7a4d9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.205861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.206556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.206832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.207576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.207785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.209638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.255876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 913/1798] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:08 2022] POST /identity/v3/projects => generated 371 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:08.258189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.277878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdbbc40-f335-4ced-831e-5665b18e4f27 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.381477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.382091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.382980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.383268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.383908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.386262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.414288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 886/1799] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:08 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:08.415487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.431855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bcc505-5a8b-4882-9a9b-ac79c77f2510 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.438472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.439432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.439848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.440223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.440975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.442719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.455368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 914/1800] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:08.459707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.471522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd4743-e605-4283-9a1c-5e05819de71b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.476371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d4ca6e-b392-4cff-a31b-6b8a1ac2589f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.476974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ca6e-b392-4cff-a31b-6b8a1ac2589f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.477239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ca6e-b392-4cff-a31b-6b8a1ac2589f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.477690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d4ca6e-b392-4cff-a31b-6b8a1ac2589f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.478385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ca6e-b392-4cff-a31b-6b8a1ac2589f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.481412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d4ca6e-b392-4cff-a31b-6b8a1ac2589f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.492789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 887/1801] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:08.497358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.507606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a14d9a8-3a08-4e5d-9ea7-2c8dcbc62150 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.513054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.513834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.514252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.514786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] PATH_INFO: `/v3/projects/1ff1227d91494582ab8eefb529e5ea60/users/f3e2cd7f3c9d48c2af9bc2ebd94ebbff/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.532618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ff1227d91494582ab8eefb529e5ea60, user_id=f3e2cd7f3c9d48c2af9bc2ebd94ebbff, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.535486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.555705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 915/1802] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:08 2022] PUT /identity/v3/projects/1ff1227d91494582ab8eefb529e5ea60/users/f3e2cd7f3c9d48c2af9bc2ebd94ebbff/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:08.568321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.568604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.568879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.619513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] MFA Rules not processed for user `f3e2cd7f3c9d48c2af9bc2ebd94ebbff`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:08.620238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:08.716944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 888/1803] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:08.722042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.770241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:08.800290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121720a-ef10-4419-b9f8-89535acbfcee admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.804995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.805602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.805881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.806153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.806828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.808717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.839602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 916/1804] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:08 2022] POST /identity/v3/projects => generated 371 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:08.840707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.852685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814106ff-cee0-4897-b1a8-5ac7d7efdcd4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.902469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:08.912582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.913322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.913607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.913880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.914548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.916463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.941222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 889/1805] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:08 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:08.942394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.952672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99dd7fef-dacf-457c-8b7b-114f75b7baf4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.958377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.959127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.959413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.959699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:08.960477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:08.962264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:08.974085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 917/1806] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:08.980047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:08.989967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75d851d-6c5e-4047-a7d7-252179f02369 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:08.994768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:08.995446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:08.995721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:08.995994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] PATH_INFO: `/v3/projects/f1fc973ce1f345dd922c9fbf954d2f40/users/5700ae7550fe4e1c83cabb5934f7afcf/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.012041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fc973ce1f345dd922c9fbf954d2f40, user_id=5700ae7550fe4e1c83cabb5934f7afcf, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:09.014683 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:09.028199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 890/1807] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:08 2022] PUT /identity/v3/projects/f1fc973ce1f345dd922c9fbf954d2f40/users/5700ae7550fe4e1c83cabb5934f7afcf/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:09.032042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:09.042605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd26d9-e4c5-4444-ae3e-526e5c96cb0a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:09.097756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535f9270-7b8b-4fa9-8cd1-93f56bc67a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:09.098383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9270-7b8b-4fa9-8cd1-93f56bc67a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:09.098658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9270-7b8b-4fa9-8cd1-93f56bc67a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:09.098923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535f9270-7b8b-4fa9-8cd1-93f56bc67a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.099550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f9270-7b8b-4fa9-8cd1-93f56bc67a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:09.101150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535f9270-7b8b-4fa9-8cd1-93f56bc67a0b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:09.111529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 918/1808] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:09 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:09.114981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:09.125086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b5c3fb-0a9f-41b6-aacb-13df547ab89f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:09.130530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:09.131290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:09.131576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:09.131852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] PATH_INFO: `/v3/projects/f1fc973ce1f345dd922c9fbf954d2f40/users/5700ae7550fe4e1c83cabb5934f7afcf/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.133749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fc973ce1f345dd922c9fbf954d2f40, user_id=5700ae7550fe4e1c83cabb5934f7afcf, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:09.136370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:09.148895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 891/1809] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:09 2022] PUT /identity/v3/projects/f1fc973ce1f345dd922c9fbf954d2f40/users/5700ae7550fe4e1c83cabb5934f7afcf/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:09.159494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:09.159720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:09.159992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.193669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] MFA Rules not processed for user `5700ae7550fe4e1c83cabb5934f7afcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:09.194927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:09.274011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 919/1810] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:09.286557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:09.287810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:09.361640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0816f643-0abd-4424-a9d3-da422f01b014 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:09.365203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd013f57-7034-4363-978f-89a76b823cb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:09.365811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013f57-7034-4363-978f-89a76b823cb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:09.366092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013f57-7034-4363-978f-89a76b823cb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:09.366482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd013f57-7034-4363-978f-89a76b823cb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.376232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013f57-7034-4363-978f-89a76b823cb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:09.377714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd013f57-7034-4363-978f-89a76b823cb5 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:09.387737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 892/1811] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:09.789545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:09.832047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:09.865267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce8b7de-5b65-4fee-b144-a1c1bf0712d5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:09.869667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:09.870314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:09.870591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:09.870860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] PATH_INFO: `/v3/users/f3e2cd7f3c9d48c2af9bc2ebd94ebbff` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.871957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3e2cd7f3c9d48c2af9bc2ebd94ebbff)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:09.873852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:09.919109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3e2cd7f3c9d48c2af9bc2ebd94ebbff'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:09.919700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3e2cd7f3c9d48c2af9bc2ebd94ebbff'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:09.927081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3e2cd7f3c9d48c2af9bc2ebd94ebbff'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:09.927725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3e2cd7f3c9d48c2af9bc2ebd94ebbff'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:09.939752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3e2cd7f3c9d48c2af9bc2ebd94ebbff'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:09.952025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 920/1812] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:09 2022] DELETE /identity/v3/users/f3e2cd7f3c9d48c2af9bc2ebd94ebbff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:09.957321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:09.957778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:09.957867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:09.976176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:09.977348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:10.036847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 893/1813] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:10.046507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:10.048622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:10.112411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a245ff49-9aa4-4b4e-a132-624225e9f55f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:10.116039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:10.116641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:10.116917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:10.117201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:10.126372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:10.127796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:10.137469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 921/1814] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:10.407638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:10.409178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:10.424340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1129208-12ab-4c64-bc5e-b6af6cae35e1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:10.430168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:10.430849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:10.431116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:10.431391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] PATH_INFO: `/v3/projects/1ff1227d91494582ab8eefb529e5ea60` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:10.432575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ff1227d91494582ab8eefb529e5ea60)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:10.434598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:10.481302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:10.487670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ff1227d91494582ab8eefb529e5ea60', 'request_id': 'req-17487bbe-54ad-43ee-956a-8455229e317c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.494612 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ff1227d91494582ab8eefb529e5ea60', 'request_id': 'req-17487bbe-54ad-43ee-956a-8455229e317c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.502901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ff1227d91494582ab8eefb529e5ea60', 'request_id': 'req-17487bbe-54ad-43ee-956a-8455229e317c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.514409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] The token cache is being invalidate because project 1ff1227d91494582ab8eefb529e5ea60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:10.514704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.515343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.517458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 894/1815] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:10 2022] DELETE /identity/v3/projects/1ff1227d91494582ab8eefb529e5ea60 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:10.521419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:10.531065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266abf48-c7f1-4da7-b956-435a3c22264d service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:10.579154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:10.590322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:10.591078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:10.591358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:10.591628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] PATH_INFO: `/v3/users/5700ae7550fe4e1c83cabb5934f7afcf` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:10.592759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5700ae7550fe4e1c83cabb5934f7afcf)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:10.595624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:10.648945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5700ae7550fe4e1c83cabb5934f7afcf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.649595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5700ae7550fe4e1c83cabb5934f7afcf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.656441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5700ae7550fe4e1c83cabb5934f7afcf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.657941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5700ae7550fe4e1c83cabb5934f7afcf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.669545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d50d856-767e-4ffd-8b6f-840e62215f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5700ae7550fe4e1c83cabb5934f7afcf'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:10.683261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 922/1816] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:10 2022] DELETE /identity/v3/users/5700ae7550fe4e1c83cabb5934f7afcf => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:10.992105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:10.993544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:11.006878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17487bbe-54ad-43ee-956a-8455229e317c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.073401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.074040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.074410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.074685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] PATH_INFO: `/v3/projects/f1fc973ce1f345dd922c9fbf954d2f40` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.075801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fc973ce1f345dd922c9fbf954d2f40)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.077726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.124646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fc973ce1f345dd922c9fbf954d2f40', 'request_id': 'req-de5c8c6a-f63d-4768-81d3-2713c5b6c832'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:11.130829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fc973ce1f345dd922c9fbf954d2f40', 'request_id': 'req-de5c8c6a-f63d-4768-81d3-2713c5b6c832'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:11.138227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fc973ce1f345dd922c9fbf954d2f40', 'request_id': 'req-de5c8c6a-f63d-4768-81d3-2713c5b6c832'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:11.149860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] The token cache is being invalidate because project f1fc973ce1f345dd922c9fbf954d2f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:11.150150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:11.150818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de5c8c6a-f63d-4768-81d3-2713c5b6c832 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:11.152306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 895/1817] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:10 2022] DELETE /identity/v3/projects/f1fc973ce1f345dd922c9fbf954d2f40 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:11.158077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.158582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.158677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.178473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:11.179744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:11.246867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 923/1818] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:11.257666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.257939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.258353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.278248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:11.279450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:11.288195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 896/1819] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:11.292885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.303834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8305c4-eca4-46db-bd1b-d2a7a2a801be None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.308710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.309664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.309664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.309911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.310648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:11.310929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.312953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.329183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 924/1820] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:11.333902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.345020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4590c244-f98f-45b3-b8a7-166c2dad33b8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.349921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.350728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.351111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.351681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.352166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.354073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.385650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 897/1821] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:11.389713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.400375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcedc17-4534-4492-a347-70355e33609b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.452373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:11.460681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.461344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.461687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.461982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.462666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.465115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.492920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 925/1822] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:11.494085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.505423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829fcd4-e9b3-41f3-948f-133da13029b7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.512954 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44140efb-098f-435b-af58-91246372b3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.513617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44140efb-098f-435b-af58-91246372b3b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.514019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44140efb-098f-435b-af58-91246372b3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.514451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44140efb-098f-435b-af58-91246372b3b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.515140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44140efb-098f-435b-af58-91246372b3b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.517342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44140efb-098f-435b-af58-91246372b3b9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.531205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 898/1823] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:11.535089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.547523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ca6096-4092-44a1-b795-894d0c265fcb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.550542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4c8c3-7bd1-4a3f-8926-c02d652858a0 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.551041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4c8c3-7bd1-4a3f-8926-c02d652858a0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.551419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4c8c3-7bd1-4a3f-8926-c02d652858a0 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.553017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 899/1824] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:21:11.556580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.557301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.558002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.558133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.558898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.561296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.561790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.562168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.566895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.588582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 926/1825] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:11 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:11.593013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.594536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:11.595777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:11.606854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea45f8-f3ea-483b-8332-39802ca4c17d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.616243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba5dccb-7329-477d-8989-f85238d09a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.617948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5dccb-7329-477d-8989-f85238d09a5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.618447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5dccb-7329-477d-8989-f85238d09a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.619316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:11.621346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba5dccb-7329-477d-8989-f85238d09a5d admin admin] PATH_INFO: `/v3/projects/ffb511e8250146bcacdaed23e46ada7b/users/fb4c70460375454caee756a257c1ef2a/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.647100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5dccb-7329-477d-8989-f85238d09a5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ffb511e8250146bcacdaed23e46ada7b, user_id=fb4c70460375454caee756a257c1ef2a, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.649747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba5dccb-7329-477d-8989-f85238d09a5d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.666259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 927/1826] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:11 2022] PUT /identity/v3/projects/ffb511e8250146bcacdaed23e46ada7b/users/fb4c70460375454caee756a257c1ef2a/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:11.676708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.676995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.677271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.690827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 900/1827] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:11.704423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.706398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:11.714711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] MFA Rules not processed for user `fb4c70460375454caee756a257c1ef2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:11.716136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:11.721512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6317af9-f244-4844-9c4a-f2e95aeeccbe None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.785759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 928/1828] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:11.790358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.806100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.806780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.807058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.807329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.818083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.819801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.833428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 901/1829] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:11.859672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee624106-c1f5-40cb-8d84-4714fff71d0d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.865257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.866093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.866404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.866676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.868268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.871640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:11.905471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 929/1830] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:11.918434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:11.931560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976184f-c8cc-4c7f-b509-6bb784cdc715 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:11.991657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:11.992285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:11.992827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:11.992827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:11.993425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:11.995334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.019160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 902/1831] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:11 2022] POST /identity/v3/users => generated 290 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:12.023172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:12.037601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35eb99bc-a19e-41bc-b83c-50175e3207ee admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:12.048826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:12.049405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.049570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.049839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.050548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:12.052151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.066893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 930/1832] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:12.067722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:12.077941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b6a5c7-d6ff-40ff-8c3a-002b32daed3b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:12.083044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:12.083701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.083973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.084242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] PATH_INFO: `/v3/projects/8c716fda95ef46ae91986e40df04b281/users/f9023ead509f4c8fbfed7cb03b9d5864/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.099627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c716fda95ef46ae91986e40df04b281, user_id=f9023ead509f4c8fbfed7cb03b9d5864, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:12.103655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.121465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 903/1833] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:12 2022] PUT /identity/v3/projects/8c716fda95ef46ae91986e40df04b281/users/f9023ead509f4c8fbfed7cb03b9d5864/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:12.126251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:12.138608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0381323d-12e6-4197-bf1a-833e6ba275bc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:12.158077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:12.219955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe78a67-fe47-438e-9397-bb03322cadd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:12.220599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78a67-fe47-438e-9397-bb03322cadd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.220875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78a67-fe47-438e-9397-bb03322cadd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.221163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe78a67-fe47-438e-9397-bb03322cadd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.222179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe78a67-fe47-438e-9397-bb03322cadd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:12.223674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe78a67-fe47-438e-9397-bb03322cadd7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.236527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 931/1834] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:12.240180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:12.246693 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:12.251058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f8d473-c238-4958-b460-5501ae85e97b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:12.260496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:12.261314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.261592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.261858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] PATH_INFO: `/v3/projects/8c716fda95ef46ae91986e40df04b281/users/f9023ead509f4c8fbfed7cb03b9d5864/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.264006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c716fda95ef46ae91986e40df04b281, user_id=f9023ead509f4c8fbfed7cb03b9d5864, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:12.266938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.284994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 904/1835] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:12 2022] PUT /identity/v3/projects/8c716fda95ef46ae91986e40df04b281/users/f9023ead509f4c8fbfed7cb03b9d5864/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:12.295293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.295504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.295777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.328245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] MFA Rules not processed for user `f9023ead509f4c8fbfed7cb03b9d5864`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:12.329582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:12.434445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 932/1836] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:12 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:12.460348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:12.462185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:12.530720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b595c971-1a65-4afe-b585-f359cd8b2ea3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:12.536898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:12.537538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.537989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.538068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.548493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:12.549851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.560593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 905/1837] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:12 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:12.632383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:12.633484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:12.646375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ed4eaf-16e4-4232-bc4a-ee8131485393 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:12.650146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20820f-c195-4f78-8088-b8bd735dc64e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:12.650866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20820f-c195-4f78-8088-b8bd735dc64e service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:12.651220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20820f-c195-4f78-8088-b8bd735dc64e service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:12.651555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20820f-c195-4f78-8088-b8bd735dc64e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:12.712890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20820f-c195-4f78-8088-b8bd735dc64e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:12.714379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20820f-c195-4f78-8088-b8bd735dc64e service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:12.724401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 933/1838] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:12 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:15.311223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:15.312701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:15.396219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ae3aa7-00af-47b6-b12e-4b859d40d71a service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:15.402161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:15.403443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:15.403899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:15.405502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] PATH_INFO: `/v3/users/a49b1016e26f41b78a6073e8c0371568` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:15.407075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49b1016e26f41b78a6073e8c0371568)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:15.409341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:15.471835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49b1016e26f41b78a6073e8c0371568'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:15.472617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49b1016e26f41b78a6073e8c0371568'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:15.479771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49b1016e26f41b78a6073e8c0371568'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:15.480480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49b1016e26f41b78a6073e8c0371568'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:15.494826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49b1016e26f41b78a6073e8c0371568'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:15.510128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 906/1839] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:15 2022] DELETE /identity/v3/users/a49b1016e26f41b78a6073e8c0371568 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:15.514584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:15.515035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:15.515247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:15.537064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:15.538367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:15.580204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:15.634175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 934/1840] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:15.645783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:15.647372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:15.664155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:15.712480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beafc66c-588d-4d69-b096-a05546fe0501 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:15.716168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:15.716768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:15.717089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:15.717359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:15.726893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:15.728310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:15.738743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 907/1841] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:16.067342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:16.068839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:16.082528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b54ef5-eb37-40fb-8706-15d7fe70310c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:16.088940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:16.090100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:16.090481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:16.090848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] PATH_INFO: `/v3/projects/b2e4d8dddc214867b6e98a31f1dabf4f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:16.095708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2e4d8dddc214867b6e98a31f1dabf4f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:16.095708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:16.168748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2e4d8dddc214867b6e98a31f1dabf4f', 'request_id': 'req-65797906-3eeb-41f0-94fd-9c466b34c6af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.175156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2e4d8dddc214867b6e98a31f1dabf4f', 'request_id': 'req-65797906-3eeb-41f0-94fd-9c466b34c6af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.181418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2e4d8dddc214867b6e98a31f1dabf4f', 'request_id': 'req-65797906-3eeb-41f0-94fd-9c466b34c6af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.195328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] The token cache is being invalidate because project b2e4d8dddc214867b6e98a31f1dabf4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:16.195637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.196249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.197744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 935/1842] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:16 2022] DELETE /identity/v3/projects/b2e4d8dddc214867b6e98a31f1dabf4f => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:16.201680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:16.206679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:16.208175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:16.212747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd048eb-0e92-4cda-aee0-bede02d37d01 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:16.282576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:16.283306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:16.283660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:16.284016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] PATH_INFO: `/v3/users/1c821aff04834c4293450d6cbdc4789a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:16.285463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c821aff04834c4293450d6cbdc4789a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:16.287440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:16.289968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65797906-3eeb-41f0-94fd-9c466b34c6af admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:16.294922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:16.295555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:16.295828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:16.296133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] PATH_INFO: `/v3/users/fb4c70460375454caee756a257c1ef2a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:16.298059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb4c70460375454caee756a257c1ef2a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:16.300074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:16.342035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c821aff04834c4293450d6cbdc4789a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.343157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c821aff04834c4293450d6cbdc4789a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.351684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c821aff04834c4293450d6cbdc4789a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.352398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c821aff04834c4293450d6cbdc4789a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.354997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb4c70460375454caee756a257c1ef2a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.356057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb4c70460375454caee756a257c1ef2a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.367341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-297c541a-691d-40f8-a8dc-89911eea8b1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c821aff04834c4293450d6cbdc4789a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.368169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb4c70460375454caee756a257c1ef2a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.368902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb4c70460375454caee756a257c1ef2a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.373834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:16.383430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 908/1843] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:16 2022] DELETE /identity/v3/users/1c821aff04834c4293450d6cbdc4789a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:16.387586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb4c70460375454caee756a257c1ef2a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:16.402183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 936/1844] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:16 2022] DELETE /identity/v3/users/fb4c70460375454caee756a257c1ef2a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:16.407386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:16.407676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:16.407944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:16.428177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:16.429366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:16.460484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:16.493184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 909/1845] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:16.507742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:16.509170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:16.576498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5504f9-e091-422d-9a2f-7da548ba65b5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:16.580740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:16.581364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:16.581638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:16.581907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:16.592414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:16.594772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:16.606485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 937/1846] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:17.063885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.065449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.080025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be9d6d-043f-496b-9d54-c562a9d2913e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.087085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.087999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.088254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.088589 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] PATH_INFO: `/v3/projects/ffb511e8250146bcacdaed23e46ada7b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.090282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ffb511e8250146bcacdaed23e46ada7b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.091857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.100697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.102531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.121921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ca51c-9334-4c65-8a61-10d01f2d050c service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.127811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.128499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.128814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.129088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] PATH_INFO: `/v3/projects/6a6a44acc6b4442aa16e706ad0242548` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.130335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a6a44acc6b4442aa16e706ad0242548)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.132349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.150423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ffb511e8250146bcacdaed23e46ada7b', 'request_id': 'req-c56adba8-8548-4b75-a667-7ac124d50f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.157180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ffb511e8250146bcacdaed23e46ada7b', 'request_id': 'req-c56adba8-8548-4b75-a667-7ac124d50f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.168452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ffb511e8250146bcacdaed23e46ada7b', 'request_id': 'req-c56adba8-8548-4b75-a667-7ac124d50f46'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.191961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a6a44acc6b4442aa16e706ad0242548', 'request_id': 'req-51ecd288-b561-4204-a5da-ec89c909c739'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.196557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] The token cache is being invalidate because project ffb511e8250146bcacdaed23e46ada7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:17.196908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.197593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.199306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 910/1847] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:17 2022] DELETE /identity/v3/projects/ffb511e8250146bcacdaed23e46ada7b => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:17.203296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a6a44acc6b4442aa16e706ad0242548', 'request_id': 'req-51ecd288-b561-4204-a5da-ec89c909c739'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.205858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.207893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.212449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a6a44acc6b4442aa16e706ad0242548', 'request_id': 'req-51ecd288-b561-4204-a5da-ec89c909c739'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.230838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] The token cache is being invalidate because project 6a6a44acc6b4442aa16e706ad0242548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:17.231161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.231931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-51ecd288-b561-4204-a5da-ec89c909c739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.233654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 938/1848] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:17 2022] DELETE /identity/v3/projects/6a6a44acc6b4442aa16e706ad0242548 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:17.238300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b5e21-2afd-441d-a148-8168d531c1f6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.238588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b5e21-2afd-441d-a148-8168d531c1f6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.238869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6b5e21-2afd-441d-a148-8168d531c1f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.262917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ae6b5e21-2afd-441d-a148-8168d531c1f6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:17.264882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ae6b5e21-2afd-441d-a148-8168d531c1f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.318429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56adba8-8548-4b75-a667-7ac124d50f46 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.339849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 939/1849] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:17.346268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.346628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.346992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.357345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:17.373025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:17.374750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.391339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 940/1850] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:17.395897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.404286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.405071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.405438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.405788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] PATH_INFO: `/v3/users/f9023ead509f4c8fbfed7cb03b9d5864` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.407247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9023ead509f4c8fbfed7cb03b9d5864)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.409452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.410378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ba2d87-a86a-49f7-ad38-ab608d32e893 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.415287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.415881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.416183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.416451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.417099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:17.417395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.419025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.419493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:17.432606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 941/1851] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:17.437166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.447924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09445ee-db22-41b3-956c-4df4413a1306 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.454454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.455133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.455528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.455808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.456453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.458379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.465373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9023ead509f4c8fbfed7cb03b9d5864'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.466038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9023ead509f4c8fbfed7cb03b9d5864'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.474884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9023ead509f4c8fbfed7cb03b9d5864'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.475553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9023ead509f4c8fbfed7cb03b9d5864'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.487255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9023ead509f4c8fbfed7cb03b9d5864'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.487888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 942/1852] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:17 2022] POST /identity/v3/projects => generated 401 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:17.492092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.493845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.500140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 911/1853] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:17 2022] DELETE /identity/v3/users/f9023ead509f4c8fbfed7cb03b9d5864 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:17.575572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f4b6c8-1481-4160-8caf-3ad1b759fcf5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.642302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.643041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.643414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.643760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.644395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.646528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.673064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 943/1854] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:17 2022] POST /identity/v3/users => generated 301 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:17.674462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.684487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a887a2-04c3-40ba-82ad-24f9a3a89551 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.689955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.690596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.690882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.691158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.691894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.693498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.706335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 912/1855] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:17.710037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.721820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5546a02-2d7c-4906-a192-44626d4a7630 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.726978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60e03a-98c1-47f7-9620-e493196cc339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.727779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60e03a-98c1-47f7-9620-e493196cc339 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.728305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60e03a-98c1-47f7-9620-e493196cc339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.728562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60e03a-98c1-47f7-9620-e493196cc339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.729704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60e03a-98c1-47f7-9620-e493196cc339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.731045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60e03a-98c1-47f7-9620-e493196cc339 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.742370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 944/1856] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:17.746614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.757847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7401164-5869-464a-be15-662f8f1f15be admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.762318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.762959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.763235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.763511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] PATH_INFO: `/v3/projects/5499ce58cab547c78f5a9a49f8f50b7d/users/a510dc8f712f401eac4c38b9c68fee43/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.779371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5499ce58cab547c78f5a9a49f8f50b7d, user_id=a510dc8f712f401eac4c38b9c68fee43, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.782030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.796189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 913/1857] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:17 2022] PUT /identity/v3/projects/5499ce58cab547c78f5a9a49f8f50b7d/users/a510dc8f712f401eac4c38b9c68fee43/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:17.808203 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.808791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.808898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.810035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.811764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.840050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] MFA Rules not processed for user `a510dc8f712f401eac4c38b9c68fee43`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:17.841911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:17.886902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa224532-be6a-4ca4-ac14-cfc4a9980ff2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.892730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:17.893366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:17.893642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:17.893915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] PATH_INFO: `/v3/projects/8c716fda95ef46ae91986e40df04b281` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:17.894140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:17.895434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c716fda95ef46ae91986e40df04b281)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:17.897257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:17.932606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 945/1858] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:17 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:17.937142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:17.949282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7a597b-2982-41ad-9edf-426f6686c352 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:17.963933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c716fda95ef46ae91986e40df04b281', 'request_id': 'req-f5bcae6a-c479-4052-b66e-428b327688a3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.966602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:17.974493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c716fda95ef46ae91986e40df04b281', 'request_id': 'req-f5bcae6a-c479-4052-b66e-428b327688a3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:17.989953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c716fda95ef46ae91986e40df04b281', 'request_id': 'req-f5bcae6a-c479-4052-b66e-428b327688a3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:18.011863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] The token cache is being invalidate because project 8c716fda95ef46ae91986e40df04b281 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:18.014505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:18.015456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f5bcae6a-c479-4052-b66e-428b327688a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:18.018865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 914/1859] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:17 2022] DELETE /identity/v3/projects/8c716fda95ef46ae91986e40df04b281 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:18.023457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cb530-8cfc-4399-a943-f402a2815908 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.024241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cb530-8cfc-4399-a943-f402a2815908 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.024797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cb530-8cfc-4399-a943-f402a2815908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.045410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.046269 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.046553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.047838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.048092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.050008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.061412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-854cb530-8cfc-4399-a943-f402a2815908 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:18.063486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-854cb530-8cfc-4399-a943-f402a2815908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:18.073437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 915/1860] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:18.082564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.082962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.083317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.084377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 946/1861] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:17 2022] POST /identity/v3/projects => generated 399 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:18.091994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.094331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:18.109415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:18.111656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:18.170418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d762f607-1b14-4f41-b184-7b11f7fe1a4f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.176132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.176807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.177188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.177563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.178301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.180344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.191929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 916/1862] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:18.197536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.210082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8eca50-0f93-44dd-9dd7-268fe6e08016 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.211977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 947/1863] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/users => generated 306 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:18.215916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.223656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.224485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.224939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.225597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.227078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:18.227579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.229604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.230914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c557123-10f8-4b7c-83f9-0c4521039454 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.269360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.271106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.271389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.271662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.272337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.274581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.292397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 917/1864] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:18.293491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.298423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 948/1865] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:18.303330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.308274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d9b02-0d44-4cc0-8bf0-2a87c49606d3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.314544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.315158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.315488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.315847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.316575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.318763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.321087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2592e404-d1ec-4410-8012-b28ea302b128 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.329202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.330129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.330450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.330731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] PATH_INFO: `/v3/projects/cbb48828c0b0428898abd5f14a283a9d/users/a5fe443aa00c4274a44ad514e0c46349/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.358932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 918/1866] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/projects => generated 379 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:18.364136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.367657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb48828c0b0428898abd5f14a283a9d, user_id=a5fe443aa00c4274a44ad514e0c46349, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.371041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.391465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 949/1867] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:18 2022] PUT /identity/v3/projects/cbb48828c0b0428898abd5f14a283a9d/users/a5fe443aa00c4274a44ad514e0c46349/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:18.395509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.424167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:18.475730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb24375-8a7d-4237-b23d-30e44298bb62 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.511260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266ef33-d1b6-4224-94ba-df6e574b1a7c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.516781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.517502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.517830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.518104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.518906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.521071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.536832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 950/1868] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:18.543103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.551957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:18.560526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fd3beb-48a4-4769-a5c4-821615108a04 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.567463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2016dc-8922-46f6-b7da-1897834d1bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.568254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2016dc-8922-46f6-b7da-1897834d1bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.568630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2016dc-8922-46f6-b7da-1897834d1bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.569033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2016dc-8922-46f6-b7da-1897834d1bdd admin admin] PATH_INFO: `/v3/projects/cbb48828c0b0428898abd5f14a283a9d/users/a5fe443aa00c4274a44ad514e0c46349/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.571087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2016dc-8922-46f6-b7da-1897834d1bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbb48828c0b0428898abd5f14a283a9d, user_id=a5fe443aa00c4274a44ad514e0c46349, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.573543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.574153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2016dc-8922-46f6-b7da-1897834d1bdd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.574375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.575024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.575024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.576068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.579758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.589933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 951/1869] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:18 2022] PUT /identity/v3/projects/cbb48828c0b0428898abd5f14a283a9d/users/a5fe443aa00c4274a44ad514e0c46349/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:18.600053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.600333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.600641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.612974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 919/1870] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/users => generated 290 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:18.617702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.640626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800dc3a2-1b73-41ab-9310-53ddef9353c1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.651521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] MFA Rules not processed for user `a5fe443aa00c4274a44ad514e0c46349`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:18.653290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:18.707243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.708131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.708407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.708677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.709351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.711087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.725731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 920/1871] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:18.729571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.736288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 952/1872] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:18.745446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc2a30-0d3b-4672-ac25-40d0509b7b7f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.750912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.751747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.752066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.752380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.753047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.755017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.764809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.766538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 921/1873] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:18.766965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:18.770537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:18.780565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5fcc41-90f1-4eb8-af59-602155f4f83b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.789083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d660ea57-c9ed-477a-bcfe-14c99bf553d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.790072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ea57-c9ed-477a-bcfe-14c99bf553d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.790658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ea57-c9ed-477a-bcfe-14c99bf553d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.790658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d660ea57-c9ed-477a-bcfe-14c99bf553d0 admin admin] PATH_INFO: `/v3/projects/d91429ae79af4197937b10ef9e8bb7b5/users/2cc12341d95a4cc7bfc62fe6a6c8aa92/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.808759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660ea57-c9ed-477a-bcfe-14c99bf553d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d91429ae79af4197937b10ef9e8bb7b5, user_id=2cc12341d95a4cc7bfc62fe6a6c8aa92, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:18.811555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d660ea57-c9ed-477a-bcfe-14c99bf553d0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:18.828076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 922/1874] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:18 2022] PUT /identity/v3/projects/d91429ae79af4197937b10ef9e8bb7b5/users/2cc12341d95a4cc7bfc62fe6a6c8aa92/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:18.840105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.842985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.842985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.845448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e4d61-c851-43e5-a405-e8acd181783d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:18.897421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] MFA Rules not processed for user `2cc12341d95a4cc7bfc62fe6a6c8aa92`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:18.898670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:18.900229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:18.916498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:18.917210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:18.917557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:18.918084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:18.999243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 923/1875] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:19.000940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.023240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.024913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.036329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:19.037731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 953/1876] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:18 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:19.091796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0961a86-0df0-47c6-bf11-95fd116b374d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.101165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.102502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.103864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.104299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.105420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.107711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.134387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.145016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497502c5-2104-4be1-9ca8-dd58ea2ab9e6 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.150171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.151051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.151497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.151907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.152020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 924/1877] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:18 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:19.154198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:19.155148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.260465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.262610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.270610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc3580-cdd2-434c-a8a2-f4e638e5e130 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.274869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 954/1878] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:19 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:19.279431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.280047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.280325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.280601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.281231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.283740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.323543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.326377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 925/1879] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:19 2022] POST /identity/v3/users => generated 290 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:19.337810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d766-036f-45a2-8f0f-7db7506c992e service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.343669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.347675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.348357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.349120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.350591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.352793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.368969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 955/1880] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:19.369948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.388507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5ff5dd-cfd7-4976-8435-bc6222c0279f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.395382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254ad26c-17aa-4079-a8e8-cec874ae82d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.397128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ad26c-17aa-4079-a8e8-cec874ae82d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.397562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ad26c-17aa-4079-a8e8-cec874ae82d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.398076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254ad26c-17aa-4079-a8e8-cec874ae82d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.398972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ad26c-17aa-4079-a8e8-cec874ae82d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.400981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254ad26c-17aa-4079-a8e8-cec874ae82d8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.416140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 926/1881] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:19 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:19.419209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.431427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0e1e55-ac6a-475a-ab3d-2a99c649aeb2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.436976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.438381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.438850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.439311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] PATH_INFO: `/v3/projects/d9ac5c24219f43e2a251130aba23dc15/users/c61c60352fab4d36b2c7df50161e37db/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.459598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ac5c24219f43e2a251130aba23dc15, user_id=c61c60352fab4d36b2c7df50161e37db, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.462300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.486639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 956/1882] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:19 2022] PUT /identity/v3/projects/d9ac5c24219f43e2a251130aba23dc15/users/c61c60352fab4d36b2c7df50161e37db/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:19.492191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e219b-a879-49bb-bc22-591753707688 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.492692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e219b-a879-49bb-bc22-591753707688 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.492819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72e219b-a879-49bb-bc22-591753707688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.545836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f72e219b-a879-49bb-bc22-591753707688 None None] MFA Rules not processed for user `c61c60352fab4d36b2c7df50161e37db`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:19.558246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f72e219b-a879-49bb-bc22-591753707688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:19.753747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 927/1883] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:19 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:19.758034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:19.911054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:19.920036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b13954-1cbb-401c-b4ef-89bdd958778a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:19.932467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:19.933406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:19.933833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:19.934308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:19.935171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:19.937567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:19.976037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 957/1884] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:19 2022] POST /identity/v3/projects => generated 377 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:19.979764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e219b-a879-49bb-bc22-591753707688 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.005755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72e219b-a879-49bb-bc22-591753707688 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.034098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:20.093109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.095449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.095943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.096288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.097007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.099902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.132248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 928/1885] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:19 2022] POST /identity/v3/users => generated 295 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:20.135830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.151465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdf9ee8-a3e2-4917-ba0b-85ab316a3512 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.160082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.160774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.161162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.161737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.162642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.164695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.206320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 958/1886] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:20.214193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.229794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cf5905-b989-4ae6-a5b8-4e98565e6df1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.235764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.236579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.237038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.237420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] PATH_INFO: `/v3/projects/78ddd119847f491eb9b7f48d92fe242e/users/78cda01210de47b29c2bccb0f64c06a8/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.262572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ddd119847f491eb9b7f48d92fe242e, user_id=78cda01210de47b29c2bccb0f64c06a8, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.265303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.281414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 929/1887] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:20 2022] PUT /identity/v3/projects/78ddd119847f491eb9b7f48d92fe242e/users/78cda01210de47b29c2bccb0f64c06a8/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:20.285965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.295914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f57e4a0-1676-4ca2-8add-78690fec408d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.353745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4c123-77be-4b49-862b-3b7a8c6095bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.354383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4c123-77be-4b49-862b-3b7a8c6095bb admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.354655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4c123-77be-4b49-862b-3b7a8c6095bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.354929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4c123-77be-4b49-862b-3b7a8c6095bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.355553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4c123-77be-4b49-862b-3b7a8c6095bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.357135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4c123-77be-4b49-862b-3b7a8c6095bb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.366953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 959/1888] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:20 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:20.371674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.383469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9fe14-9ea3-4121-a979-6c1dcd2813ca admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.389220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.389868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.390161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.390451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] PATH_INFO: `/v3/projects/78ddd119847f491eb9b7f48d92fe242e/users/78cda01210de47b29c2bccb0f64c06a8/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.393227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78ddd119847f491eb9b7f48d92fe242e, user_id=78cda01210de47b29c2bccb0f64c06a8, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.395843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.407371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 930/1889] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:20 2022] PUT /identity/v3/projects/78ddd119847f491eb9b7f48d92fe242e/users/78cda01210de47b29c2bccb0f64c06a8/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:20.418513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.418869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.419282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.451410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] MFA Rules not processed for user `78cda01210de47b29c2bccb0f64c06a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:20.452827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:20.536481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 960/1890] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:20.548608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.550863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:20.612496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a77712a-12d0-49f5-befe-eea617dfe430 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.622701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.623320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.623599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.624237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.636218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.638495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.649505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 931/1891] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:20 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:20.699966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:20.711126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10924ca-6c81-467e-b20b-69607dda3914 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:20.715219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e3d78-86f0-4fef-bd6e-6a714528e6ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:20.715831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3d78-86f0-4fef-bd6e-6a714528e6ab service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:20.716105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3d78-86f0-4fef-bd6e-6a714528e6ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:20.716371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e3d78-86f0-4fef-bd6e-6a714528e6ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:20.765868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:20.780125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e3d78-86f0-4fef-bd6e-6a714528e6ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:20.781563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e3d78-86f0-4fef-bd6e-6a714528e6ab service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:20.793234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 961/1892] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:20 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:22.846491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:22.849148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:22.969708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5dfae9-9a17-494f-828a-189290034f06 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:22.980409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:22.981109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:22.981656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:22.981937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] PATH_INFO: `/v3/users/29dc579ad4cd4417b843f6c96c505d0a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:22.985887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29dc579ad4cd4417b843f6c96c505d0a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:22.997496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:23.043635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:23.144319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29dc579ad4cd4417b843f6c96c505d0a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:23.145743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29dc579ad4cd4417b843f6c96c505d0a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:23.167092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29dc579ad4cd4417b843f6c96c505d0a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:23.171029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29dc579ad4cd4417b843f6c96c505d0a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:23.213192 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29dc579ad4cd4417b843f6c96c505d0a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:23.246411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 932/1893] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:22 2022] DELETE /identity/v3/users/29dc579ad4cd4417b843f6c96c505d0a => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:23.252287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:23.252846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:23.252846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:23.294566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:23.298408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:23.473328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 962/1894] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:23.513762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:23.516155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:23.654367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5da6006-3f2d-47a8-8218-c06ed27d14f6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:23.662048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:23.662753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:23.663042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:23.663287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:23.689908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:23.691452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:23.706321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 933/1895] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:23.998502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:24.000142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.015191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:24.017419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.023162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8064fb04-0e16-4b25-b74d-3575ea565bf9 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:24.031702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:24.033682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.034060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.034463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] PATH_INFO: `/v3/users/2cc12341d95a4cc7bfc62fe6a6c8aa92` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.036478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151f5eac-f3b8-451f-a1d6-94fd6c2ecc24 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:24.036640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cc12341d95a4cc7bfc62fe6a6c8aa92)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:24.039151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:24.043987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:24.044921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.045583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.046246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] PATH_INFO: `/v3/users/a510dc8f712f401eac4c38b9c68fee43` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.048185 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a510dc8f712f401eac4c38b9c68fee43)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:24.051445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:24.128657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cc12341d95a4cc7bfc62fe6a6c8aa92'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.131186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cc12341d95a4cc7bfc62fe6a6c8aa92'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.153618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cc12341d95a4cc7bfc62fe6a6c8aa92'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.157088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cc12341d95a4cc7bfc62fe6a6c8aa92'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.178384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a510dc8f712f401eac4c38b9c68fee43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.179306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4df0dbed-cdca-4ba1-8dc3-b5418b4b7682 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cc12341d95a4cc7bfc62fe6a6c8aa92'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.179427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a510dc8f712f401eac4c38b9c68fee43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.191385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a510dc8f712f401eac4c38b9c68fee43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.192275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a510dc8f712f401eac4c38b9c68fee43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.199217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 963/1896] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:23 2022] DELETE /identity/v3/users/2cc12341d95a4cc7bfc62fe6a6c8aa92 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:24.204912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.205459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.206265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.212573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2043526c-39b1-47df-80e3-668fc1a5481b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a510dc8f712f401eac4c38b9c68fee43'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.238387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 934/1897] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:24 2022] DELETE /identity/v3/users/a510dc8f712f401eac4c38b9c68fee43 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:24.239055 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:24.240479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.240641 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.240792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.241071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.270414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:24.272259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.282319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:24.387859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 964/1898] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:24.397103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 935/1899] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:24.401824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:24.404100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.411291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:24.418454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:24.425334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.559533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4dae5b-4422-4df5-b52d-24245e6e0aaf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:24.567786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:24.568469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.568709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.568988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.595718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:24.596921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea982d6-0b36-4caf-8bf6-463dad1756b5 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:24.597215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:24.601921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:24.609993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.612483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.612872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.632847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:24.633068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 965/1900] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:24.633068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:24.633243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:24.637887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:24.646036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 936/1901] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:24.659412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e746b-a77e-49f5-9fcd-b5a8ebb05bbe service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:24.678574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:24.679369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:24.679727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:24.680080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] PATH_INFO: `/v3/projects/0572b836ca474904991e034ee199c1f3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:24.681484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0572b836ca474904991e034ee199c1f3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:24.683528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:24.774121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0572b836ca474904991e034ee199c1f3', 'request_id': 'req-b17c95c2-4022-4f55-9707-a3cec1865825'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.792358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0572b836ca474904991e034ee199c1f3', 'request_id': 'req-b17c95c2-4022-4f55-9707-a3cec1865825'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.807759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0572b836ca474904991e034ee199c1f3', 'request_id': 'req-b17c95c2-4022-4f55-9707-a3cec1865825'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.830803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] The token cache is being invalidate because project 0572b836ca474904991e034ee199c1f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:24.831102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.832465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:24.837473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 966/1902] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:24 2022] DELETE /identity/v3/projects/0572b836ca474904991e034ee199c1f3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:24.841541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:24.861232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e22d951-cb4d-4b2e-94fa-7f1957beb0f8 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:25.083799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:25.084399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:25.084511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:25.084769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] PATH_INFO: `/v3/users/a058c94040a94880bb31cdb088872073` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:25.086405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] RBAC: Authorizing `identity:delete_user(user_id=a058c94040a94880bb31cdb088872073)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:25.090857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:25.199429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a058c94040a94880bb31cdb088872073'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.201650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a058c94040a94880bb31cdb088872073'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.213050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:25.216288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a058c94040a94880bb31cdb088872073'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.218594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a058c94040a94880bb31cdb088872073'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.249065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a058c94040a94880bb31cdb088872073'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.282090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 937/1903] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:24 2022] DELETE /identity/v3/users/a058c94040a94880bb31cdb088872073 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:25.303426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:25.305286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.322148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c95c2-4022-4f55-9707-a3cec1865825 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:25.408446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:25.409073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:25.409360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:25.409632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] PATH_INFO: `/v3/users/fc211049c85d4072b0d725074c4c457f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:25.411178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc211049c85d4072b0d725074c4c457f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:25.412760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:25.460239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:25.483166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc211049c85d4072b0d725074c4c457f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.483888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc211049c85d4072b0d725074c4c457f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.496801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc211049c85d4072b0d725074c4c457f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.503271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc211049c85d4072b0d725074c4c457f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.520029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9ccfd0c4-7ecb-47eb-ad51-00876743b0ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc211049c85d4072b0d725074c4c457f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.538122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:25.539763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.555581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 967/1904] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:25 2022] DELETE /identity/v3/users/fc211049c85d4072b0d725074c4c457f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:25.559442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d69de3-6845-422a-8ff0-61703cd0ecee admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:25.560484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:25.560888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:25.561226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:25.584219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:25.585847 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.633884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:25.634790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:25.635126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:25.635407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] PATH_INFO: `/v3/projects/d91429ae79af4197937b10ef9e8bb7b5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:25.636625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d91429ae79af4197937b10ef9e8bb7b5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:25.639020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:25.690389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 968/1905] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:25.691539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd91429ae79af4197937b10ef9e8bb7b5', 'request_id': 'req-3b78ac34-b7cb-4ea4-a40c-98730801e486'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.694708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:25.696639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.698336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd91429ae79af4197937b10ef9e8bb7b5', 'request_id': 'req-3b78ac34-b7cb-4ea4-a40c-98730801e486'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.707942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd91429ae79af4197937b10ef9e8bb7b5', 'request_id': 'req-3b78ac34-b7cb-4ea4-a40c-98730801e486'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.722865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] The token cache is being invalidate because project d91429ae79af4197937b10ef9e8bb7b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:25.723300 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.724482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.729229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 938/1906] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:25 2022] DELETE /identity/v3/projects/d91429ae79af4197937b10ef9e8bb7b5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:25.731259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:25.732826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.763983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c581584-2d0c-455d-b3f3-736c201eb6b1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:25.817566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b78ac34-b7cb-4ea4-a40c-98730801e486 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:25.817776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:25.818522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:25.818802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:25.819090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:25.821305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.823765 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:25.824412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:25.824713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:25.824987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] PATH_INFO: `/v3/users/c61c60352fab4d36b2c7df50161e37db` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:25.826149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c61c60352fab4d36b2c7df50161e37db)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:25.828118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:25.831356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:25.834816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:25.836457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:25.848833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 969/1907] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:25.885477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c61c60352fab4d36b2c7df50161e37db'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.886142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c61c60352fab4d36b2c7df50161e37db'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.897275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c61c60352fab4d36b2c7df50161e37db'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.898180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c61c60352fab4d36b2c7df50161e37db'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.919685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c61c60352fab4d36b2c7df50161e37db'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:25.940203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 939/1908] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:25 2022] DELETE /identity/v3/users/c61c60352fab4d36b2c7df50161e37db => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:25.958675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:25.960341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:25.980553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f282d107-039c-4c02-af6d-2308804f1179 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:25.989129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:26.138247 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:26.142391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.142391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.142391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] PATH_INFO: `/v3/projects/5499ce58cab547c78f5a9a49f8f50b7d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.143278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5499ce58cab547c78f5a9a49f8f50b7d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:26.145255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:26.196464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:26.197974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.223604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28584-d64f-4169-9741-d62f54cbac04 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:26.227753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5499ce58cab547c78f5a9a49f8f50b7d', 'request_id': 'req-55d28063-3f11-401f-81a1-d0d51192fd02'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.253100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5499ce58cab547c78f5a9a49f8f50b7d', 'request_id': 'req-55d28063-3f11-401f-81a1-d0d51192fd02'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.271128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5499ce58cab547c78f5a9a49f8f50b7d', 'request_id': 'req-55d28063-3f11-401f-81a1-d0d51192fd02'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.301221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] The token cache is being invalidate because project 5499ce58cab547c78f5a9a49f8f50b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:26.302006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.303049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.310261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 970/1909] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:25 2022] DELETE /identity/v3/projects/5499ce58cab547c78f5a9a49f8f50b7d => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:26.312415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:26.317342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.329105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:26.329852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.330730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.330730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] PATH_INFO: `/v3/projects/cb6b60f6fb264d20a50f52f99c99d210` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.332425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb6b60f6fb264d20a50f52f99c99d210)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:26.337442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:26.466749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb6b60f6fb264d20a50f52f99c99d210', 'request_id': 'req-ba85cecc-bfc4-4d71-b262-027a931ef4cb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.480584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d28063-3f11-401f-81a1-d0d51192fd02 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:26.483017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb6b60f6fb264d20a50f52f99c99d210', 'request_id': 'req-ba85cecc-bfc4-4d71-b262-027a931ef4cb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.497298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb6b60f6fb264d20a50f52f99c99d210', 'request_id': 'req-ba85cecc-bfc4-4d71-b262-027a931ef4cb'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.513865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] The token cache is being invalidate because project cb6b60f6fb264d20a50f52f99c99d210 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:26.514539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.515387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ba85cecc-bfc4-4d71-b262-027a931ef4cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.518376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 940/1910] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:26 2022] DELETE /identity/v3/projects/cb6b60f6fb264d20a50f52f99c99d210 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:26.523347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.523675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.523967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.581166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:26.586719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:26.588867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.603851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:26.622582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:26.624237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.626903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.627522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] PATH_INFO: `/v3/users/a5fe443aa00c4274a44ad514e0c46349` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.633426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5fe443aa00c4274a44ad514e0c46349)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:26.635641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:26.697011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 941/1911] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:26.697011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:26.699347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.714940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f223cb2-5390-4477-a8c0-293e35d8e9f9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:26.721055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:26.721779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.722067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.722386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] PATH_INFO: `/v3/projects/369bc7ac4b1c4b58afc1b1addf818797` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.724188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=369bc7ac4b1c4b58afc1b1addf818797)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:26.726706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:26.736990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5fe443aa00c4274a44ad514e0c46349'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.738086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5fe443aa00c4274a44ad514e0c46349'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.756446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5fe443aa00c4274a44ad514e0c46349'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.757223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5fe443aa00c4274a44ad514e0c46349'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.776487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5fe443aa00c4274a44ad514e0c46349'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.790032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '369bc7ac4b1c4b58afc1b1addf818797', 'request_id': 'req-66be986b-9906-42b9-91e0-88ff482ae1a1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.792478 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 971/1912] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:26 2022] DELETE /identity/v3/users/a5fe443aa00c4274a44ad514e0c46349 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:26.792686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:26.798011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.803148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '369bc7ac4b1c4b58afc1b1addf818797', 'request_id': 'req-66be986b-9906-42b9-91e0-88ff482ae1a1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.820234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '369bc7ac4b1c4b58afc1b1addf818797', 'request_id': 'req-66be986b-9906-42b9-91e0-88ff482ae1a1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.832890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] The token cache is being invalidate because project 369bc7ac4b1c4b58afc1b1addf818797 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:26.833676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.834568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-66be986b-9906-42b9-91e0-88ff482ae1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:26.836529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 942/1913] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:26 2022] DELETE /identity/v3/projects/369bc7ac4b1c4b58afc1b1addf818797 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:26.837903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.838322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.838743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.863606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:26.865161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.950034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 943/1914] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:26.950034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:26.952393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:26.957064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7038d7-2600-4512-a657-71f43177a299 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:26.968310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:26.969555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.972017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3dd8ec-4a42-4c3f-adfe-ef9ea7a9904e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:26.982795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:26.982943 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.983141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] PATH_INFO: `/v3/projects/d9ac5c24219f43e2a251130aba23dc15` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.983841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:26.984222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:26.984685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] PATH_INFO: `/v3/users/57e4399f676447ab9d82e8111b015264` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:26.986737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] RBAC: Authorizing `identity:delete_user(user_id=57e4399f676447ab9d82e8111b015264)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:26.988776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ac5c24219f43e2a251130aba23dc15)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:26.989755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:26.990785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.032849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:27.052760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57e4399f676447ab9d82e8111b015264'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.053929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57e4399f676447ab9d82e8111b015264'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.063239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57e4399f676447ab9d82e8111b015264'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.064825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57e4399f676447ab9d82e8111b015264'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.076625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ac5c24219f43e2a251130aba23dc15', 'request_id': 'req-8afda0fe-e53b-4a5b-99c6-64a7291808dc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.083960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57e4399f676447ab9d82e8111b015264'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.091744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ac5c24219f43e2a251130aba23dc15', 'request_id': 'req-8afda0fe-e53b-4a5b-99c6-64a7291808dc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.106016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 944/1915] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:26 2022] DELETE /identity/v3/users/57e4399f676447ab9d82e8111b015264 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:27.106016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.106016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.114919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ac5c24219f43e2a251130aba23dc15', 'request_id': 'req-8afda0fe-e53b-4a5b-99c6-64a7291808dc'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.128435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:27.161748 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] The token cache is being invalidate because project d9ac5c24219f43e2a251130aba23dc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:27.162562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.162801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.168762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 972/1916] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:26 2022] DELETE /identity/v3/projects/d9ac5c24219f43e2a251130aba23dc15 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:27.169731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.172571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.242590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423bc49-6019-4c51-94d1-9d15a81ec6af admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.248842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.249507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.249791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.250092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.253665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:27.253665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.255358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.270690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 945/1917] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:27.271613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.273532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.275951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afda0fe-e53b-4a5b-99c6-64a7291808dc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.281523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.284708 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.285079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.285408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] PATH_INFO: `/v3/projects/cbb48828c0b0428898abd5f14a283a9d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.287637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbb48828c0b0428898abd5f14a283a9d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.289609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.300762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec49c8d4-7b6b-4f51-a99f-21410c47e062 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.309623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.310779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.311125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.311512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] PATH_INFO: `/v3/users/78cda01210de47b29c2bccb0f64c06a8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.312771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78cda01210de47b29c2bccb0f64c06a8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.317069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.357460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbb48828c0b0428898abd5f14a283a9d', 'request_id': 'req-f8905638-4649-465e-9c6d-a3614a42fe46'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.366521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbb48828c0b0428898abd5f14a283a9d', 'request_id': 'req-f8905638-4649-465e-9c6d-a3614a42fe46'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.375522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbb48828c0b0428898abd5f14a283a9d', 'request_id': 'req-f8905638-4649-465e-9c6d-a3614a42fe46'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.398591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78cda01210de47b29c2bccb0f64c06a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.399350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78cda01210de47b29c2bccb0f64c06a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.408034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78cda01210de47b29c2bccb0f64c06a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.408800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78cda01210de47b29c2bccb0f64c06a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.409408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] The token cache is being invalidate because project cbb48828c0b0428898abd5f14a283a9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:27.409783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.410550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.412291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 973/1918] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:27 2022] DELETE /identity/v3/projects/cbb48828c0b0428898abd5f14a283a9d => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:27.416404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.417877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.424666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d36d5484-86fb-4f63-b0f5-1deb38c125e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78cda01210de47b29c2bccb0f64c06a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.449590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 946/1919] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:27 2022] DELETE /identity/v3/users/78cda01210de47b29c2bccb0f64c06a8 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:27.450946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.451243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.451492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.483123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:27.487873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.506517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:27.567668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8905638-4649-465e-9c6d-a3614a42fe46 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.588274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 947/1920] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:27.589767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.592793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.622050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:27.625576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5a9a69-62e4-4c97-87e5-dc9f280c69b0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.633523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.634128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.634748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.635029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] PATH_INFO: `/v3/projects/c0261626d1a241ecaa9478dacebddb75` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.636385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0261626d1a241ecaa9478dacebddb75)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.639365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.673052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4831bb5-e224-4378-a08d-f650314542dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.674378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4831bb5-e224-4378-a08d-f650314542dd admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.674848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4831bb5-e224-4378-a08d-f650314542dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.675195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4831bb5-e224-4378-a08d-f650314542dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.676129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4831bb5-e224-4378-a08d-f650314542dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.679432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4831bb5-e224-4378-a08d-f650314542dd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.722524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 974/1921] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/projects => generated 387 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:27.723986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c9525-9c86-4222-abb9-f6f1ac05633d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.724344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c9525-9c86-4222-abb9-f6f1ac05633d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.724715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6c9525-9c86-4222-abb9-f6f1ac05633d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.731368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0261626d1a241ecaa9478dacebddb75', 'request_id': 'req-f14d3458-dad7-4091-9193-8c8e4ff4a802'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.746764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0261626d1a241ecaa9478dacebddb75', 'request_id': 'req-f14d3458-dad7-4091-9193-8c8e4ff4a802'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.748712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3a6c9525-9c86-4222-abb9-f6f1ac05633d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:27.750408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3a6c9525-9c86-4222-abb9-f6f1ac05633d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.759357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0261626d1a241ecaa9478dacebddb75', 'request_id': 'req-f14d3458-dad7-4091-9193-8c8e4ff4a802'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.777075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] The token cache is being invalidate because project c0261626d1a241ecaa9478dacebddb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:27.779103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.779623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:27.784040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 948/1922] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:27 2022] DELETE /identity/v3/projects/c0261626d1a241ecaa9478dacebddb75 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:27.789706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.791640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.851308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 975/1923] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:27.856705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.857036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.857372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.882623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d3458-dad7-4091-9193-8c8e4ff4a802 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.887974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451673b1-1749-4394-98eb-56d02e05b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.889056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451673b1-1749-4394-98eb-56d02e05b461 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.889535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451673b1-1749-4394-98eb-56d02e05b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.889837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:27.890186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451673b1-1749-4394-98eb-56d02e05b461 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.891055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451673b1-1749-4394-98eb-56d02e05b461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.891329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.893546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451673b1-1749-4394-98eb-56d02e05b461 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.910436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 976/1924] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:27.910941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.927121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 949/1925] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/users => generated 294 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:27.928527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.928805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.929173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.935521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f34b5-7286-4854-b3c4-90c3a4f59ead None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.942889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.943499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.943775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.944040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.944685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:27.944987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:27.946850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:27.964410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 977/1926] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:27.965280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.969283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:27.975428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:27.985978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f1d0d0-ea91-4bc8-8bd7-e2455dd5f981 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:27.990645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 950/1927] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:27.993524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:27.994716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:27.995309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:27.995560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:27.995941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:27.997257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.000442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.015643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 978/1928] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:28.015643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.025689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3679746-b597-438c-b4dd-af4f8e26ea41 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.026497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02db7b48-68d9-4cae-a0f7-2dd46923cd07 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.033287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.033944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.034250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.034518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.035163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:28.035458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.039193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.039941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.040056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.040781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.040781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.041456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.043844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.070712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 979/1929] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:28.071752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.075045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:28.085949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6fab9-1dca-4343-95b5-a9d1ddb2eda0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.090259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 951/1930] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:27 2022] POST /identity/v3/projects => generated 367 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:28.090259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.090939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.092981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.093581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.094111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.094414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.096227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.097821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.113595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 980/1931] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:28.113595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.122722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72d7c61-6d07-4626-9444-3d883285008a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.158703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ee23c1-46d3-4f8b-bde3-104b94480895 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.163280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.163894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.164286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.164558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] PATH_INFO: `/v3/projects/78ddd119847f491eb9b7f48d92fe242e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.166012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78ddd119847f491eb9b7f48d92fe242e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.168062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.180316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.180968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.181248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.181687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.182370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.184281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.190342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:28.220321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 981/1932] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:28.220321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.221812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78ddd119847f491eb9b7f48d92fe242e', 'request_id': 'req-999b4110-f09b-4182-a264-5a96f1f298a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:28.227766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78ddd119847f491eb9b7f48d92fe242e', 'request_id': 'req-999b4110-f09b-4182-a264-5a96f1f298a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:28.236530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78ddd119847f491eb9b7f48d92fe242e', 'request_id': 'req-999b4110-f09b-4182-a264-5a96f1f298a8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:28.248001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] The token cache is being invalidate because project 78ddd119847f491eb9b7f48d92fe242e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:28.248579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:28.248930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:28.255153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 952/1933] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:28 2022] DELETE /identity/v3/projects/78ddd119847f491eb9b7f48d92fe242e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:28.255153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.255153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.293822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c590829-82e5-478d-94f9-71a1c7034a17 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.299266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.299891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.300163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.300503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.301141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.303112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.331558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 982/1934] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:28.331558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.332853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.337733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999b4110-f09b-4182-a264-5a96f1f298a8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.343697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665716c6-6974-4c1d-affe-fb9118a4f6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.345231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665716c6-6974-4c1d-affe-fb9118a4f6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.345609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665716c6-6974-4c1d-affe-fb9118a4f6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.345889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665716c6-6974-4c1d-affe-fb9118a4f6ff admin admin] PATH_INFO: `/v3/projects/01ff12c049ff4334be521be69a1d7445/users/69413ae24cfc421e9293e8d1b6e65e44/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.355697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c10e40e-543b-4633-8159-4ced347fd5fc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.364195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.364842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.365158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.365473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.365576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665716c6-6974-4c1d-affe-fb9118a4f6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=01ff12c049ff4334be521be69a1d7445, user_id=69413ae24cfc421e9293e8d1b6e65e44, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.366219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.368210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665716c6-6974-4c1d-affe-fb9118a4f6ff admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.368405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.382827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 953/1935] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:28 2022] PUT /identity/v3/projects/01ff12c049ff4334be521be69a1d7445/users/69413ae24cfc421e9293e8d1b6e65e44/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:28.384327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334858be-c153-4ec7-924f-4cc8b305c84d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.384591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334858be-c153-4ec7-924f-4cc8b305c84d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.384863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334858be-c153-4ec7-924f-4cc8b305c84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.401870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 983/1936] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/users => generated 288 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:28.402756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.406373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.410162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-334858be-c153-4ec7-924f-4cc8b305c84d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:28.411372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-334858be-c153-4ec7-924f-4cc8b305c84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.416717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:28.420164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f09fdf-c44f-4764-b175-b774787190f0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.478230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 954/1937] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:28.485656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.485656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.485656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.485931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.485931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.485931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.485931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.486359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.487730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.501438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 984/1938] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:28.502420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.509934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] MFA Rules not processed for user `69413ae24cfc421e9293e8d1b6e65e44`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:28.510823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.512498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670fcd11-f28a-4fbc-8a40-35550832ad4c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.523818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e0cc43-7d58-4698-827c-0b238cb08bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.524522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cc43-7d58-4698-827c-0b238cb08bea admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.524874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cc43-7d58-4698-827c-0b238cb08bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.525211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e0cc43-7d58-4698-827c-0b238cb08bea admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.525923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0cc43-7d58-4698-827c-0b238cb08bea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.527700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e0cc43-7d58-4698-827c-0b238cb08bea admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.541549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 985/1939] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:28.541549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.541549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.541549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.560272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:28.561484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:28.570121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 986/1940] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:28.574257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.585339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b0ec3-5063-4833-889d-272a89ea18a3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.591062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.591868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.592520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.592827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.593645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.595570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 955/1941] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:28.595963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.596456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.613026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 987/1942] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:28.613026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.613236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57f387-34d1-43a6-a285-2f4a6c4b421f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.620452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.621066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.621343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.621613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.621840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362055d1-c472-4bc7-8a14-03869b7b6650 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.622427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.624192 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.632754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.632754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.632754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.632754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.633179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:28.633179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.633179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.635677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 956/1943] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:28.636858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.647252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b506ba-fca8-4c3b-9413-65eda58c5558 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.653883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 988/1944] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:28.653883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.657546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.658157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.658460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.658728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.659354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.661263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.669975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824c59d-4784-48d7-a9c5-24fb24c4dffb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.679855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.681542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.682304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.682739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] PATH_INFO: `/v3/projects/01f8953a541149969502d6eff6bc8eff/users/b193935309c54abda3adf68bf0875fb7/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.696415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 957/1945] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:28.697174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.699080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:28.720565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01f8953a541149969502d6eff6bc8eff, user_id=b193935309c54abda3adf68bf0875fb7, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.723352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.752397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 989/1946] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:28 2022] PUT /identity/v3/projects/01f8953a541149969502d6eff6bc8eff/users/b193935309c54abda3adf68bf0875fb7/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:28.754097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.838472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c77b7-c246-4584-b2dd-66fb1970d4e3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.853691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:28.891981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a6aaf0-a574-497a-83e0-7989046a15f1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:28.911521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.912234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.913036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.913231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.914100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.916865 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.921457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08606d5-6c15-4a33-a534-5adfced443fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:28.922132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08606d5-6c15-4a33-a534-5adfced443fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.922495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08606d5-6c15-4a33-a534-5adfced443fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.922845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08606d5-6c15-4a33-a534-5adfced443fe admin admin] PATH_INFO: `/v3/projects/a5184d1afd824ee3a0e9a640239c249c/users/bda3a904463c413c81ff90d51e2e9057/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:28.949677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08606d5-6c15-4a33-a534-5adfced443fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5184d1afd824ee3a0e9a640239c249c, user_id=bda3a904463c413c81ff90d51e2e9057, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:28.954143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08606d5-6c15-4a33-a534-5adfced443fe admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:28.962808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 990/1947] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/projects => generated 373 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:28.963789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:28.983430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 958/1948] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:28 2022] PUT /identity/v3/projects/a5184d1afd824ee3a0e9a640239c249c/users/bda3a904463c413c81ff90d51e2e9057/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:28.985014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:28.986989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:28.988717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.033100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] MFA Rules not processed for user `b193935309c54abda3adf68bf0875fb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:29.034631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:29.057132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbf3fc1-dc94-4db3-a853-5006c15f3299 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.117968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 959/1949] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:29.122248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.149232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84305e7-01a4-4e17-aab9-441eade98945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.150701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84305e7-01a4-4e17-aab9-441eade98945 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.150986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84305e7-01a4-4e17-aab9-441eade98945 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.151338 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84305e7-01a4-4e17-aab9-441eade98945 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.151918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84305e7-01a4-4e17-aab9-441eade98945 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.155146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84305e7-01a4-4e17-aab9-441eade98945 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.180993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 991/1950] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:28 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:29.182136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.182472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.182741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.186622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fccfb5d-1911-4d89-b42e-c3007f1e2113 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.193653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.194485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.194758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.195034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.195658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.197559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.204607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:29.216583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] MFA Rules not processed for user `bda3a904463c413c81ff90d51e2e9057`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:29.219325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:29.224320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 960/1951] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:29.225122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.235804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ccebe-942e-43a4-ab6e-d0e27fb48244 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.240600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.241201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.241557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.241832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.242508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.244440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.262160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:29.277527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 961/1952] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:29.278384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.303317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 992/1953] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:29.305210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.316803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b900f8dd-275c-4eb1-86ff-0f1fd9d85bfe None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.343726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8994838d-e9c0-4fc0-9041-0203b29fc76c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.348945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.349541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.349823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.350092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.350773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.352341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.363440 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 962/1954] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.364882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.374297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea191e2-1806-49ee-86a4-304cf4421fec admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.375556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.376746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.377018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.377288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.377926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.379626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.379762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.380205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.380483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.380768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.381365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.383285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.389463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 993/1955] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.390268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.400273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff2f49-dc4e-4d1b-a14e-fef994261c61 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.405019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.405597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.408801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 963/1956] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/users => generated 290 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:29.408926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.408926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.409055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.409141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.409141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.416307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8489c72d-d544-4edb-b65a-58b8a8e19df5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.420893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.421525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.421800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.422081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] PATH_INFO: `/v3/projects/e450c181ca2d4894bd47d48626914725/users/979f4de8bccc44f3b03e23e0baeb29d2/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.436481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 994/1957] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:29.438095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e450c181ca2d4894bd47d48626914725, user_id=979f4de8bccc44f3b03e23e0baeb29d2, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.440862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.441366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.454512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 964/1958] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:29 2022] PUT /identity/v3/projects/e450c181ca2d4894bd47d48626914725/users/979f4de8bccc44f3b03e23e0baeb29d2/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:29.455066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.511145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612ad152-364a-43a9-8d9b-0e232414f8b9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.515698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.516296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.517179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.517449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.518084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.518813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c45415-e1d5-4d5f-8a28-6b60a2bbda9f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.519704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.523381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.523979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.524252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.524661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.525297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.526898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.529722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 995/1959] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.530357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.540106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 965/1960] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.540106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.540364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3615b87-fbca-4b06-a1e1-6842e6df1b0d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.544860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.545446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.545722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.545999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.546657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.548726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.549474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bbf3e5-87fd-49e3-b713-5a2f51a7ead0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.553901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.554507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.554781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.555050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.555665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.557210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.558703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:29.569772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 966/1961] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.570591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.575755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 996/1962] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:29.575755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.582258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2d4f0-434f-4afa-9f28-78fd764e43b3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.584382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7067c839-b2a8-407e-81e6-b6f2d3212c35 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.586777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.587409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.587684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.587953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] PATH_INFO: `/v3/projects/883abe2a7db046a4989991886424299f/users/87ba75f413154a9b943b6d6b781dc011/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.589088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.589718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.590001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.590304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] PATH_INFO: `/v3/projects/58c993907d944dc7980c35d419291bc1/users/0062ac7c3ff14caaaebea8804725bace/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.603158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=883abe2a7db046a4989991886424299f, user_id=87ba75f413154a9b943b6d6b781dc011, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.605347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c993907d944dc7980c35d419291bc1, user_id=0062ac7c3ff14caaaebea8804725bace, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.605775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.609198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.628244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 997/1963] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:29 2022] PUT /identity/v3/projects/58c993907d944dc7980c35d419291bc1/users/0062ac7c3ff14caaaebea8804725bace/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:29.629218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.635784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 967/1964] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:29 2022] PUT /identity/v3/projects/883abe2a7db046a4989991886424299f/users/87ba75f413154a9b943b6d6b781dc011/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:29.636882 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.654919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:29.729472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf60a7-72cd-4122-bda3-6a63a88cac6a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.732275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ce2873-2366-4178-b461-4020eeda66cf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.734415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbafc1c9-82a4-453b-920e-25b92f3905b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.735069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc1c9-82a4-453b-920e-25b92f3905b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.735996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc1c9-82a4-453b-920e-25b92f3905b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.736251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbafc1c9-82a4-453b-920e-25b92f3905b6 admin admin] PATH_INFO: `/v3/projects/e450c181ca2d4894bd47d48626914725/users/979f4de8bccc44f3b03e23e0baeb29d2/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.737904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.738148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc1c9-82a4-453b-920e-25b92f3905b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e450c181ca2d4894bd47d48626914725, user_id=979f4de8bccc44f3b03e23e0baeb29d2, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.738528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.738804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.739073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.739734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.740872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbafc1c9-82a4-453b-920e-25b92f3905b6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.741297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.751147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 968/1965] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.751943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.754544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 998/1966] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:29 2022] PUT /identity/v3/projects/e450c181ca2d4894bd47d48626914725/users/979f4de8bccc44f3b03e23e0baeb29d2/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:29.755821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13911-983a-44ba-b8b1-602ce88ed58c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.756106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13911-983a-44ba-b8b1-602ce88ed58c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.756378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a13911-983a-44ba-b8b1-602ce88ed58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.763927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec6ccb-64b6-4e31-9206-55f8dd9af242 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.786330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-09a13911-983a-44ba-b8b1-602ce88ed58c None None] MFA Rules not processed for user `87ba75f413154a9b943b6d6b781dc011`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:29.787535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-09a13911-983a-44ba-b8b1-602ce88ed58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:29.820249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.820859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.821135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.821404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.822037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.823640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.833204 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 969/1967] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:29.834054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.843466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e178ff9-e348-472e-be28-71f7965bf9b0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.848820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.849442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.849719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.849994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] PATH_INFO: `/v3/projects/b60f6f591c684c26bfcb6b791d511e12/users/417399ada57f44b49c2539ca2fc65383/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.851908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 999/1968] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:29.854173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.854497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.854768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.866097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60f6f591c684c26bfcb6b791d511e12, user_id=417399ada57f44b49c2539ca2fc65383, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.868760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.883158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] MFA Rules not processed for user `979f4de8bccc44f3b03e23e0baeb29d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:29.883431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 970/1969] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:29 2022] PUT /identity/v3/projects/b60f6f591c684c26bfcb6b791d511e12/users/417399ada57f44b49c2539ca2fc65383/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:29.884219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.887153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:29.928797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:29.945671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275897fa-0aad-4bbc-9c23-9b9e811bbc5d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:29.950812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:29.951466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:29.951744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:29.952067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] PATH_INFO: `/v3/projects/58c993907d944dc7980c35d419291bc1/users/0062ac7c3ff14caaaebea8804725bace/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:29.954700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58c993907d944dc7980c35d419291bc1, user_id=0062ac7c3ff14caaaebea8804725bace, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:29.957376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:29.958852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1000/1970] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:29 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:29.959676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.960995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:29.968483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 971/1971] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:29 2022] PUT /identity/v3/projects/58c993907d944dc7980c35d419291bc1/users/0062ac7c3ff14caaaebea8804725bace/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:29.968483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:29.984959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:30.032357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd647e07-6a0c-48ae-a027-78dbb34a5fe5 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.034946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846ffd4-b73c-47e1-9912-98c6665c2328 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.040002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.040652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.040935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.041214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.041871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.043543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.059286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 972/1972] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:30.059286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:30.060227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.090782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe5ffb-c03d-4ae1-903f-18b1e1436974 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.091413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5ffb-c03d-4ae1-903f-18b1e1436974 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.091668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5ffb-c03d-4ae1-903f-18b1e1436974 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.091937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe5ffb-c03d-4ae1-903f-18b1e1436974 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.127523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c7d1-43a8-4aa3-9a10-70c4df95d31d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.133818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.134467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.134748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.135023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.158338 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe5ffb-c03d-4ae1-903f-18b1e1436974 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.159828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fe5ffb-c03d-4ae1-903f-18b1e1436974 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.171180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1001/1973] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:30.172068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289365-a09e-4908-885c-be440aa6b381 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.172368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289365-a09e-4908-885c-be440aa6b381 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.173315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72289365-a09e-4908-885c-be440aa6b381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.206755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-72289365-a09e-4908-885c-be440aa6b381 None None] MFA Rules not processed for user `0062ac7c3ff14caaaebea8804725bace`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:30.208097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-72289365-a09e-4908-885c-be440aa6b381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.220702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.224206 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.250475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 973/1974] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:30.252227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:30.266417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:30.277863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45256817-68cb-457c-a8b8-0a8ced6becb5 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.283799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.284893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.287962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.288441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] PATH_INFO: `/v3/projects/b60f6f591c684c26bfcb6b791d511e12/users/417399ada57f44b49c2539ca2fc65383/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.291370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b60f6f591c684c26bfcb6b791d511e12, user_id=417399ada57f44b49c2539ca2fc65383, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.295082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.315735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 974/1975] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:30 2022] PUT /identity/v3/projects/b60f6f591c684c26bfcb6b791d511e12/users/417399ada57f44b49c2539ca2fc65383/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:30.316604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:30.346027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1002/1976] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:30 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:30.346027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.346027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.346027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.412168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] MFA Rules not processed for user `417399ada57f44b49c2539ca2fc65383`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:30.414742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.437613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:30.478865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33805646-195b-4a3d-873a-0637678ed5f8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.491404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.492064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.492328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.492596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] PATH_INFO: `/v3/users/69413ae24cfc421e9293e8d1b6e65e44` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.495634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69413ae24cfc421e9293e8d1b6e65e44)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.497725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.516605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1003/1977] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:30 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:30.518470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:30.608924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69413ae24cfc421e9293e8d1b6e65e44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:30.609943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69413ae24cfc421e9293e8d1b6e65e44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:30.649071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69413ae24cfc421e9293e8d1b6e65e44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:30.649642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69413ae24cfc421e9293e8d1b6e65e44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:30.671157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afea7d-86bc-42a2-b89c-ef35ca5eb6b3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.675866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.676483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.676758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.677030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.678672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69413ae24cfc421e9293e8d1b6e65e44'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:30.678979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.715980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 975/1978] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:30 2022] DELETE /identity/v3/users/69413ae24cfc421e9293e8d1b6e65e44 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:30.719262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:30.723748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.762632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.764142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb53515-faf8-49b2-b5c3-f6d27aaca0eb service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.787940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1004/1979] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:30.789443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.789776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.790007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.816055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96041f92-4f96-43f2-809f-849c74b53e21 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:30.821056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:30.821753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:30.822102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:30.822432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:30.824365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.824880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:30.826081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.838513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:30.931031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1005/1980] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:30.937705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:30.939982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:30.940769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:30.942181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:30.958000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 976/1981] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:31.028330 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a220364-c2a2-4ea9-91d3-c724d768e9a6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:31.033618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:31.035075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:31.036084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:31.036581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:31.058900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:31.060666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:31.084023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1006/1982] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:31.178837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:31.180335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:31.192816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3aa97ab-c84e-4425-b5ff-168537340c8d service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:31.196439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:31.197124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:31.197421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:31.197679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:31.199545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:31.220052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:31.285864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:31.287958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:31.299652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 977/1983] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:31.504147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:31.512527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:31.547390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bad7199-e593-4a0e-80d6-50cac2ff61ef service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:31.564471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:31.567197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:31.568405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:31.570154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] PATH_INFO: `/v3/projects/01ff12c049ff4334be521be69a1d7445` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:31.572711 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01ff12c049ff4334be521be69a1d7445)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:31.576677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:31.664732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01ff12c049ff4334be521be69a1d7445', 'request_id': 'req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.672461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01ff12c049ff4334be521be69a1d7445', 'request_id': 'req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.683679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01ff12c049ff4334be521be69a1d7445', 'request_id': 'req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.706933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] The token cache is being invalidate because project 01ff12c049ff4334be521be69a1d7445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:31.707415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.713347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.715196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1007/1984] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:31 2022] DELETE /identity/v3/projects/01ff12c049ff4334be521be69a1d7445 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:31.720037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:31.733677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3743d0-b318-40ac-be87-c0f420853234 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:31.815425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:31.816096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:31.816382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:31.816657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] PATH_INFO: `/v3/users/979f4de8bccc44f3b03e23e0baeb29d2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:31.817818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=979f4de8bccc44f3b03e23e0baeb29d2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:31.819875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:31.869137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '979f4de8bccc44f3b03e23e0baeb29d2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.869892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '979f4de8bccc44f3b03e23e0baeb29d2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.877031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '979f4de8bccc44f3b03e23e0baeb29d2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.877716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '979f4de8bccc44f3b03e23e0baeb29d2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.890267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2997023c-1ae7-4616-9517-a49e6bdc1f04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '979f4de8bccc44f3b03e23e0baeb29d2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:31.915922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 978/1985] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:31 2022] DELETE /identity/v3/users/979f4de8bccc44f3b03e23e0baeb29d2 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:32.377506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:32.388982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:32.411266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55802f02-8c7d-4bdb-8ce4-fb60f4269a76 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:32.457140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:32.487885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:32.488598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:32.488966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:32.489312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] PATH_INFO: `/v3/projects/e450c181ca2d4894bd47d48626914725` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:32.490673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e450c181ca2d4894bd47d48626914725)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:32.492714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:32.566971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e450c181ca2d4894bd47d48626914725', 'request_id': 'req-94cfd97e-82b7-453c-997c-7a94755d6875'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:32.575269 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e450c181ca2d4894bd47d48626914725', 'request_id': 'req-94cfd97e-82b7-453c-997c-7a94755d6875'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:32.582011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e450c181ca2d4894bd47d48626914725', 'request_id': 'req-94cfd97e-82b7-453c-997c-7a94755d6875'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:32.595937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] The token cache is being invalidate because project e450c181ca2d4894bd47d48626914725 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:32.596356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:32.597058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-94cfd97e-82b7-453c-997c-7a94755d6875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:32.598816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1008/1986] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:32 2022] DELETE /identity/v3/projects/e450c181ca2d4894bd47d48626914725 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:32.603510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:32.604027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:32.604124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:32.625007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:32.629071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:32.678689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:32.714421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 979/1987] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:32.722630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:32.723106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:32.723257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:32.743991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:32.745126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:32.755164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1009/1988] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:32.759438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:32.772296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aeec9-c20e-4d78-a21f-f77168fd247f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:32.783227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:32.783864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:32.784145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:32.785969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:32.785969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:32.785969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:32.787560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:32.806254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 980/1989] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:32.810734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:32.821939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a1e687-bd66-4195-9dbd-6d61a1ea538d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:32.827991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:32.829623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:32.830045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:32.830492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:32.831854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:32.837220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:32.893690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1010/1990] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:32 2022] POST /identity/v3/projects => generated 385 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:32.896014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:32.908549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97912de1-3d2e-45b1-a431-b593d2d31f34 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:32.999834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.000416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.000800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.001184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.002019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.004106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.029415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 981/1991] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:32 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:33.034437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.045824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc000e-521e-4a74-a94b-1da337b6f3b3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.052002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.052735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.053084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.053423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.054137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.055855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.068499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1011/1992] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:33.073673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.087578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfebd09-b886-4dd6-a20f-2ba9b62c95b9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.093649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.095292 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.095949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.096640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.097719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.099483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.108453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.110334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.114819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 982/1993] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:33.116130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.124872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b544b87e-6ee9-4ff5-944b-c8a5a690b000 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.128293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82760854-ad24-4ccc-8f74-84159501de70 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.131758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.132766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.133270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.133771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] PATH_INFO: `/v3/users/3bb0f21252f34df18d1c932e56687aa2` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.133969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cde0dc-a87a-46f4-add3-5ef7036296b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.134701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cde0dc-a87a-46f4-add3-5ef7036296b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.135135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cde0dc-a87a-46f4-add3-5ef7036296b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.135323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cde0dc-a87a-46f4-add3-5ef7036296b1 admin admin] PATH_INFO: `/v3/projects/65014d4189534c99a881b07d1f5c1f1d/users/eef6e19e9dce4502999e83cec4f49eb6/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.136348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3bb0f21252f34df18d1c932e56687aa2)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.140095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.154303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cde0dc-a87a-46f4-add3-5ef7036296b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65014d4189534c99a881b07d1f5c1f1d, user_id=eef6e19e9dce4502999e83cec4f49eb6, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.157982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cde0dc-a87a-46f4-add3-5ef7036296b1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.177666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 983/1994] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:33 2022] PUT /identity/v3/projects/65014d4189534c99a881b07d1f5c1f1d/users/eef6e19e9dce4502999e83cec4f49eb6/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:33.189471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16828676-d33a-48e8-8c00-4f11105128de None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.190854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16828676-d33a-48e8-8c00-4f11105128de None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.191405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16828676-d33a-48e8-8c00-4f11105128de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.207650 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3bb0f21252f34df18d1c932e56687aa2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.208533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3bb0f21252f34df18d1c932e56687aa2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.222593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3bb0f21252f34df18d1c932e56687aa2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.223441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3bb0f21252f34df18d1c932e56687aa2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.234487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-16828676-d33a-48e8-8c00-4f11105128de None None] MFA Rules not processed for user `eef6e19e9dce4502999e83cec4f49eb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:33.236514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-16828676-d33a-48e8-8c00-4f11105128de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.239409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f8c31f86-bbb8-4de3-b9bd-b1e1a076c8c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3bb0f21252f34df18d1c932e56687aa2'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.248523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:33.258746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1012/1995] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:33 2022] DELETE /identity/v3/users/3bb0f21252f34df18d1c932e56687aa2 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:33.260575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.261439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.262965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.286745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:33.288319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.316489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 984/1996] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:33.320942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16828676-d33a-48e8-8c00-4f11105128de None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.322286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-16828676-d33a-48e8-8c00-4f11105128de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.333667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:33.351938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1013/1997] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:33.362326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.363754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.398578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16828676-d33a-48e8-8c00-4f11105128de None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.403630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.404238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.404513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.404785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.405405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.407866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.436434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f8402-20f5-4877-a17d-fa462072226f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.441815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.442467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.442726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.442998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.445815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 985/1998] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:33 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:33.450148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.453452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.455394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.471715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1014/1999] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:33.562413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7dfcd-ec9a-4ded-baa1-d32e24a1442b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.571498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.571498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.571889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.572049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.572685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.575108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.605917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 986/2000] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:33 2022] POST /identity/v3/users => generated 300 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:33.607341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.625395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a8d5e-edf1-41f7-a73c-2d8e9b4eda7e service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.631053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.637939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.639363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.639363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.639363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.642516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.654941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1015/2001] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:33.659712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.677045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0748ddfd-f448-43e0-8e38-68927784390d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.682505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.683287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.683562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.683831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] PATH_INFO: `/v3/projects/606780a7ef2c44079ad5a6101c24243d/users/57110f5a98fd450cb16e23e8642b5d72/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.700085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=606780a7ef2c44079ad5a6101c24243d, user_id=57110f5a98fd450cb16e23e8642b5d72, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.702748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.718304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 987/2002] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:33 2022] PUT /identity/v3/projects/606780a7ef2c44079ad5a6101c24243d/users/57110f5a98fd450cb16e23e8642b5d72/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:33.723106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.733769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1ede9-0dfc-4994-b9b6-a6f1f894ef04 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.780450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.782913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.800325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.800828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.801103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.801373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.802012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.803734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.816700 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1016/2003] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:33.820697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.830829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f430b325-f660-4a12-a14a-d4740bdbf08f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.835753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d838ca88-f279-4c64-86a4-652c2f564698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.836426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838ca88-f279-4c64-86a4-652c2f564698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.836704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838ca88-f279-4c64-86a4-652c2f564698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.836978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d838ca88-f279-4c64-86a4-652c2f564698 admin admin] PATH_INFO: `/v3/projects/606780a7ef2c44079ad5a6101c24243d/users/57110f5a98fd450cb16e23e8642b5d72/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.838902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838ca88-f279-4c64-86a4-652c2f564698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=606780a7ef2c44079ad5a6101c24243d, user_id=57110f5a98fd450cb16e23e8642b5d72, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.841651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d838ca88-f279-4c64-86a4-652c2f564698 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.853482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1017/2004] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:33 2022] PUT /identity/v3/projects/606780a7ef2c44079ad5a6101c24243d/users/57110f5a98fd450cb16e23e8642b5d72/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:33.855752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b4b7a-08e0-4dc7-9636-d5e63c413bbb admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:33.864432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:33.865709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.866894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.867052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.892191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:33.895101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] MFA Rules not processed for user `57110f5a98fd450cb16e23e8642b5d72`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:33.897071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.920533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:33.921170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:33.921452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:33.921731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] PATH_INFO: `/v3/projects/4360ce98d2db41fb90c378f2df1d653e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:33.923498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4360ce98d2db41fb90c378f2df1d653e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:33.925526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:33.968204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1018/2005] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:33.975384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4360ce98d2db41fb90c378f2df1d653e', 'request_id': 'req-bf1ba858-b465-4b10-ab3c-c457a2100b61'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.979790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:33.981137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:33.981341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4360ce98d2db41fb90c378f2df1d653e', 'request_id': 'req-bf1ba858-b465-4b10-ab3c-c457a2100b61'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:33.996924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4360ce98d2db41fb90c378f2df1d653e', 'request_id': 'req-bf1ba858-b465-4b10-ab3c-c457a2100b61'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.015318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] The token cache is being invalidate because project 4360ce98d2db41fb90c378f2df1d653e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:34.015610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.016662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.018403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 988/2006] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:33 2022] DELETE /identity/v3/projects/4360ce98d2db41fb90c378f2df1d653e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:34.022222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:34.023757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.044869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5273992b-24e3-47df-9f51-baf61bd31bf1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.090855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1ba858-b465-4b10-ab3c-c457a2100b61 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.093367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.094262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.095072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.095517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.096548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.097168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.097380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.097473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.097699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] PATH_INFO: `/v3/users/5cda7bc3d33944a9b31e09ff7bdf189d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.098835 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cda7bc3d33944a9b31e09ff7bdf189d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.100764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.142285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:34.148523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cda7bc3d33944a9b31e09ff7bdf189d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.149202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cda7bc3d33944a9b31e09ff7bdf189d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.158354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cda7bc3d33944a9b31e09ff7bdf189d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.159016 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cda7bc3d33944a9b31e09ff7bdf189d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.165507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.167345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.172024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cda7bc3d33944a9b31e09ff7bdf189d'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.176840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1019/2007] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:34.177679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:34.178971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.185816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 989/2008] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:34 2022] DELETE /identity/v3/users/5cda7bc3d33944a9b31e09ff7bdf189d => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:34.192781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:34.196701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbeaf50b-f2d7-4d30-a4b5-839c53a8d2db service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.258913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:34.261409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.289851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.290580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.290860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.291336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] PATH_INFO: `/v3/users/87ba75f413154a9b943b6d6b781dc011` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.292254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87ba75f413154a9b943b6d6b781dc011)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.294179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.373940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87ba75f413154a9b943b6d6b781dc011'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.375232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87ba75f413154a9b943b6d6b781dc011'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.379170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeb8bc1-d4c1-48bb-ad60-0b986ff6307e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.384345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.385213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.385576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.386145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] PATH_INFO: `/v3/users/eef6e19e9dce4502999e83cec4f49eb6` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.387227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eef6e19e9dce4502999e83cec4f49eb6)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.387481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87ba75f413154a9b943b6d6b781dc011'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.388190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87ba75f413154a9b943b6d6b781dc011'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.389362 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.411386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c2fb15fe-c8b1-4e49-b362-53735ca07436 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87ba75f413154a9b943b6d6b781dc011'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.431873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1020/2009] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:34 2022] DELETE /identity/v3/users/87ba75f413154a9b943b6d6b781dc011 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:34.436587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.437095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.437262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.450810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eef6e19e9dce4502999e83cec4f49eb6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.451572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eef6e19e9dce4502999e83cec4f49eb6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.461128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:34.462441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.465733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eef6e19e9dce4502999e83cec4f49eb6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.466600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eef6e19e9dce4502999e83cec4f49eb6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.489008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e8c89ce6-cf9b-4f9e-a220-d2a212ad7ff7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eef6e19e9dce4502999e83cec4f49eb6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.514941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 990/2010] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:34 2022] DELETE /identity/v3/users/eef6e19e9dce4502999e83cec4f49eb6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:34.521251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.521918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.522229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.549868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1021/2011] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:34.563434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:34.564238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:34.565457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.566090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.584229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce39286-6306-4823-a4bb-5e452f416f30 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.605833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:34.621668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:34.663343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 991/2012] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:34.664086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:34.665569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.683272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.684573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.685466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.685874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] PATH_INFO: `/v3/projects/6f8d4f5fdf5c4393914bbe2d30cfb855` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.687151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f8d4f5fdf5c4393914bbe2d30cfb855)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.689415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.753521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10e866-2f8e-4f20-9d31-ce629e40395f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.767506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f8d4f5fdf5c4393914bbe2d30cfb855', 'request_id': 'req-78febe67-afef-4dd7-892c-d420e8a368a7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.780626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f8d4f5fdf5c4393914bbe2d30cfb855', 'request_id': 'req-78febe67-afef-4dd7-892c-d420e8a368a7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.788169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f8d4f5fdf5c4393914bbe2d30cfb855', 'request_id': 'req-78febe67-afef-4dd7-892c-d420e8a368a7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.812640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] The token cache is being invalidate because project 6f8d4f5fdf5c4393914bbe2d30cfb855 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:34.812972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.814278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:34.816242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1022/2013] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:34 2022] DELETE /identity/v3/projects/6f8d4f5fdf5c4393914bbe2d30cfb855 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:34.817246 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:34.818661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.825595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.826444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.826924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.827338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.830437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.892272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78febe67-afef-4dd7-892c-d420e8a368a7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:34.897539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:34.898472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.898761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.899033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.900840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.910848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.912530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc471ce-d530-4bb5-acea-ebc63d7974bd service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.915355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:34.916802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20482d5-f67c-41d3-befe-0bbac6392896 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:34.924810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 992/2014] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:34.925133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd732631-1807-4405-a313-1d386666a1ac None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.925423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd732631-1807-4405-a313-1d386666a1ac None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.925697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd732631-1807-4405-a313-1d386666a1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.927973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1023/2015] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:34.946669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bd732631-1807-4405-a313-1d386666a1ac None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:34.947891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bd732631-1807-4405-a313-1d386666a1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:34.956480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 993/2016] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:34.964087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:34.964450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:34.964911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:34.990810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:34.992035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.003580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1024/2017] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:35.008609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd732631-1807-4405-a313-1d386666a1ac None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.025571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd732631-1807-4405-a313-1d386666a1ac None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.037284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.038154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.038527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.038923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.039729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:35.040116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.043026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.064703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 994/2018] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:35.069666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.079255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3655d17-76c4-4b04-b1d1-51acb73af8cd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.084098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.084768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.085044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.085422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.086152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.088138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.122337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1025/2019] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:35.123680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.133967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1c2d9-10f6-4948-bd03-8f26951bd8a5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.153458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:35.203497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.205168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.205168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.205384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.206079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.208098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.240189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 995/2020] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/users => generated 283 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:35.244303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.260115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb9b539-5b07-43b9-ad60-87da92905dbb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.264957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.265919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.265919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.266115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.268208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.270679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.285466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1026/2021] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:35.286952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.302115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c883ad60-3287-46b0-9ed6-9c81f1ce1a12 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.310660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.311553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.312178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.312683 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.314142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.316647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.325835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.328020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.334649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 996/2022] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:35.339442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.351469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6899f0ea-216b-4b69-80ee-aa1fd6987a4b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.357557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.358271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.358642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.358840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] PATH_INFO: `/v3/projects/12c1e25efed7429290f625660c601d53/users/9728d169eee14008a954b521c2e02735/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.367518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977208d6-954f-4758-91ea-4208852b6be6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.373154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.374103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.374661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.375172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] PATH_INFO: `/v3/projects/65014d4189534c99a881b07d1f5c1f1d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.375293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] RBAC: Authorizing `identity:create_grant(project_id=12c1e25efed7429290f625660c601d53, user_id=9728d169eee14008a954b521c2e02735, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.377154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=65014d4189534c99a881b07d1f5c1f1d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.379084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.380092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.394137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:35.395797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 997/2023] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:35 2022] PUT /identity/v3/projects/12c1e25efed7429290f625660c601d53/users/9728d169eee14008a954b521c2e02735/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:35.399221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.402631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.431952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65014d4189534c99a881b07d1f5c1f1d', 'request_id': 'req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.439277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65014d4189534c99a881b07d1f5c1f1d', 'request_id': 'req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.445344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65014d4189534c99a881b07d1f5c1f1d', 'request_id': 'req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.458628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] The token cache is being invalidate because project 65014d4189534c99a881b07d1f5c1f1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:35.458946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.459584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-c295b54a-6fb7-43e4-a754-4f7c342bbf8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.461240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1027/2024] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:35 2022] DELETE /identity/v3/projects/65014d4189534c99a881b07d1f5c1f1d => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:35.462462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.462864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.463144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.478464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2d0fd-95a5-4d6f-b2da-753ef72e119d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.494956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] MFA Rules not processed for user `9728d169eee14008a954b521c2e02735`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:35.496178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.534992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.535951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.536230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.536510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] PATH_INFO: `/v3/projects/883abe2a7db046a4989991886424299f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.537624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=883abe2a7db046a4989991886424299f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.539560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.568993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1028/2025] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:35.572899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:35.573400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.575137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.589362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d1727-5cc3-4c9f-8d4e-683ab91c3906 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.591152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '883abe2a7db046a4989991886424299f', 'request_id': 'req-acf7c6e9-da44-4a73-86af-170fe989d979'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.598344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '883abe2a7db046a4989991886424299f', 'request_id': 'req-acf7c6e9-da44-4a73-86af-170fe989d979'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.606081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '883abe2a7db046a4989991886424299f', 'request_id': 'req-acf7c6e9-da44-4a73-86af-170fe989d979'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.618660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] The token cache is being invalidate because project 883abe2a7db046a4989991886424299f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:35.618946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.619670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.621229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 998/2026] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:35 2022] DELETE /identity/v3/projects/883abe2a7db046a4989991886424299f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:35.622002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.623308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.660948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.661585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.661864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.662135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] PATH_INFO: `/v3/users/57110f5a98fd450cb16e23e8642b5d72` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.663302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57110f5a98fd450cb16e23e8642b5d72)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.665250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.692207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf7c6e9-da44-4a73-86af-170fe989d979 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.697822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ceb0d-d0e0-4db3-a8b1-dfb10325b031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.698510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ceb0d-d0e0-4db3-a8b1-dfb10325b031 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.698866 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ceb0d-d0e0-4db3-a8b1-dfb10325b031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.699049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ceb0d-d0e0-4db3-a8b1-dfb10325b031 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.699569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:35.699743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ceb0d-d0e0-4db3-a8b1-dfb10325b031 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.701685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115ceb0d-d0e0-4db3-a8b1-dfb10325b031 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.714852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57110f5a98fd450cb16e23e8642b5d72'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.715555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57110f5a98fd450cb16e23e8642b5d72'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.724761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57110f5a98fd450cb16e23e8642b5d72'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.725475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57110f5a98fd450cb16e23e8642b5d72'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.735847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 999/2027] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:35.735847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f243d-2c77-4517-bdc1-43c78c713cf9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.735847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f243d-2c77-4517-bdc1-43c78c713cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.735847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f243d-2c77-4517-bdc1-43c78c713cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.737992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57110f5a98fd450cb16e23e8642b5d72'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:35.751406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1029/2028] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:35 2022] DELETE /identity/v3/users/57110f5a98fd450cb16e23e8642b5d72 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:35.754004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-654f243d-2c77-4517-bdc1-43c78c713cf9 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:35.755477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.755564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-654f243d-2c77-4517-bdc1-43c78c713cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.756518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.822958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1000/2029] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:35.830642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.830929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.831194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.839843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c1736-4796-4681-b7c6-27646af04240 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.845767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.846553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.846932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.847302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.848042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.850387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.852231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:35.853477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:35.863208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1001/2030] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:35.867674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.881092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c1fb42-0e8e-49db-9faf-6e3acf1899ee None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.882685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1030/2031] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/users => generated 288 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:35.885948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.886077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.886715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.887678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.887969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.888614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:35.889072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.891951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.904976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1002/2032] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:21:35.909818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.910801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4240b-c0c0-4f8e-9685-30ebf74df880 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.918596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.919219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.919506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.919769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.920414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.921142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29f79bd-b113-4a20-9ebe-5416d38d4631 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.922063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.927233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.927818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.928099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.928368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.929431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.931674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:35.935147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1031/2033] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:35 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:35.939737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.953459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce39b32d-030d-428c-835e-0a3bd3bbe104 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:35.958894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:35.959725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:35.960705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:35.961180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:35.961508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] PATH_INFO: `/v3/projects/2b8cd6efab3f435ca138c3d7636eae01/users/70aadf6fd6604066a4a46c829992bf36/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:35.975979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1003/2034] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:35.977441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:35.983507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8cd6efab3f435ca138c3d7636eae01, user_id=70aadf6fd6604066a4a46c829992bf36, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:35.987521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.014470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1032/2035] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:35 2022] PUT /identity/v3/projects/2b8cd6efab3f435ca138c3d7636eae01/users/70aadf6fd6604066a4a46c829992bf36/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:36.016403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.136656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa928b1-7074-433c-9831-f56b10529762 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.166737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d8f357-8f00-4578-b982-f71884b7fa38 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.173234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.174931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.174931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.174931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.175497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.177100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.183970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:36.192360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1033/2036] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:36.193527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.195836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.222025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3d1cc-feee-4798-8d82-bee6a2e568a8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.228554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.229309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.229601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.229971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] PATH_INFO: `/v3/projects/606780a7ef2c44079ad5a6101c24243d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.231641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=606780a7ef2c44079ad5a6101c24243d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.233470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.244456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.247453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.247453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.247453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.247453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.264088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.290278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1004/2037] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:35 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:36.291325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.301471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '606780a7ef2c44079ad5a6101c24243d', 'request_id': 'req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:36.306035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0540d3b-48e9-4f7c-bc5e-325e84d28d18 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.320449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '606780a7ef2c44079ad5a6101c24243d', 'request_id': 'req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:36.333561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '606780a7ef2c44079ad5a6101c24243d', 'request_id': 'req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:36.351058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] The token cache is being invalidate because project 606780a7ef2c44079ad5a6101c24243d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:36.351752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:36.351997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:36.353507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1034/2038] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:36 2022] DELETE /identity/v3/projects/606780a7ef2c44079ad5a6101c24243d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:36.355073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.356449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.393810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53b5b09-41c3-465c-9f66-3bb52a4ea2cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.395347 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5b09-41c3-465c-9f66-3bb52a4ea2cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.395730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5b09-41c3-465c-9f66-3bb52a4ea2cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.396093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53b5b09-41c3-465c-9f66-3bb52a4ea2cc admin admin] PATH_INFO: `/v3/projects/2b8cd6efab3f435ca138c3d7636eae01/users/70aadf6fd6604066a4a46c829992bf36/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.399468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b5b09-41c3-465c-9f66-3bb52a4ea2cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2b8cd6efab3f435ca138c3d7636eae01, user_id=70aadf6fd6604066a4a46c829992bf36, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.403175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53b5b09-41c3-465c-9f66-3bb52a4ea2cc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.420120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1005/2039] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:36 2022] PUT /identity/v3/projects/2b8cd6efab3f435ca138c3d7636eae01/users/70aadf6fd6604066a4a46c829992bf36/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:36.421479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af19e1-5297-4da9-a988-603b835a492a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.421827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af19e1-5297-4da9-a988-603b835a492a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.422233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3af19e1-5297-4da9-a988-603b835a492a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.459238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a3af19e1-5297-4da9-a988-603b835a492a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:36.460468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a3af19e1-5297-4da9-a988-603b835a492a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.480497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93081a-9e3f-4570-ad7f-c51a7dc8ea2f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.491315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:36.543716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1006/2040] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:36.545264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.545756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.546255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.590501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff071eb9-df1a-4659-9741-fdf931ecfbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.591638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071eb9-df1a-4659-9741-fdf931ecfbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.591958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] MFA Rules not processed for user `70aadf6fd6604066a4a46c829992bf36`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:36.593248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.594437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071eb9-df1a-4659-9741-fdf931ecfbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.595150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff071eb9-df1a-4659-9741-fdf931ecfbc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.596180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff071eb9-df1a-4659-9741-fdf931ecfbc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.598382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff071eb9-df1a-4659-9741-fdf931ecfbc4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.628806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1035/2041] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:36.630315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.633800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.634180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.658281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:36.659555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.670023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1036/2042] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:36.675093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.686752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96b1ec-2b77-42e2-a97c-8a2e8d5a1aa4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.691952 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.692551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.692827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.693097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.693602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1007/2043] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:36.694309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.694761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.697319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.706380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962f16bf-bc19-4992-b228-a701f7445ade None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.710412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1037/2044] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:36 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:36.711245 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.711746 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.712088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.712430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.712741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.713532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:36.713923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.715282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.715556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.728537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1008/2045] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:36.728737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:36.732664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.743288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1382eb-4d42-46da-ab3c-234963f4b6c1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.753054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.753725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.754002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.754494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] PATH_INFO: `/v3/projects/8e031c5a09954ec986fd00aab53512ab/users/04b102ea49b74b8e901aea1d9b507cc0/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.774602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e031c5a09954ec986fd00aab53512ab, user_id=04b102ea49b74b8e901aea1d9b507cc0, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.777373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.784109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfd4a5-79db-4379-9563-4db10a6c7b70 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.787805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d456bf-5736-4f5f-9a00-c485bb38605b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.788406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d456bf-5736-4f5f-9a00-c485bb38605b service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.788680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d456bf-5736-4f5f-9a00-c485bb38605b service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.788942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d456bf-5736-4f5f-9a00-c485bb38605b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.791249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1009/2046] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:36 2022] PUT /identity/v3/projects/8e031c5a09954ec986fd00aab53512ab/users/04b102ea49b74b8e901aea1d9b507cc0/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:36.800240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.800370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d456bf-5736-4f5f-9a00-c485bb38605b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.800925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d456bf-5736-4f5f-9a00-c485bb38605b service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.812607 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1038/2047] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:36.814192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.814518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.814875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.849279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] MFA Rules not processed for user `04b102ea49b74b8e901aea1d9b507cc0`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:36.850866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.871294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d7e2af-0409-4a4e-86c5-135dd5a92f34 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.876905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.877648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.877928 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.878219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.878865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:36.880776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:36.915225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1010/2048] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:36 2022] POST /identity/v3/projects => generated 383 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:36.916244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.923165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:36.923999 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1039/2049] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:36 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:36.925331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:36.937928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131972c7-780a-456e-bc16-c19fd47b4cbd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.976812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894f7f2-6d96-47cc-8cab-6425b5acdd63 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:36.980428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.981050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.981324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.981591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.983531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:36.996004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:36.996686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:36.997067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:36.997405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:36.998154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.000105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.022255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1040/2050] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:36 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:37.026754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.038226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c34941-e682-425d-99fc-b31d598c1f57 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.042894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.043476 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.043750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.044020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.044651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.046294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.048077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.049898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.056397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1041/2051] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:37.060312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.061903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1011/2052] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:36 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:37.071350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8af592e-f333-4f4b-b84b-10c750be5015 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.076179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ff1d8-411a-406e-aa93-1bb0cf35f082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.076777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ff1d8-411a-406e-aa93-1bb0cf35f082 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.077052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ff1d8-411a-406e-aa93-1bb0cf35f082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.077323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ff1d8-411a-406e-aa93-1bb0cf35f082 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.077963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ff1d8-411a-406e-aa93-1bb0cf35f082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.079656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ff1d8-411a-406e-aa93-1bb0cf35f082 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.093574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1042/2053] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:37.095554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.107459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6cfc0a-4471-409c-92fb-6420f0ea4c1b service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.112824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.113984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.114369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.114647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] PATH_INFO: `/v3/projects/bc2b312831c8475caf68f2e9a9fe6e24/users/e67a5ab134894180a7f3a6943b5b7991/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.133226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc2b312831c8475caf68f2e9a9fe6e24, user_id=e67a5ab134894180a7f3a6943b5b7991, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.135962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.152552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1012/2054] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:37 2022] PUT /identity/v3/projects/bc2b312831c8475caf68f2e9a9fe6e24/users/e67a5ab134894180a7f3a6943b5b7991/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:37.164970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.165284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.165555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.176966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.202060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:37.213751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] MFA Rules not processed for user `e67a5ab134894180a7f3a6943b5b7991`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:37.215018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:37.253972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dd4c13-82fd-4dd4-88bf-4e441bfd06d6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.259059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.260181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.260559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.260926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.296898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1043/2055] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:37.298454 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.326109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:37.339675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.341150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.351258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1013/2056] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:37.368950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c667b90-e798-4b69-88e4-56056dcd80cc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.373570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.374191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.374497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.374772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.375932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.377605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.406371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1044/2057] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:37 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:37.411225 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.422903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237990a2-88a9-4fcb-a07a-bcda7f5a2ed9 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.475463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.477167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:37.499323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.499939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.500216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.500497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.501340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.504117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.527061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1014/2058] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:37 2022] POST /identity/v3/users => generated 298 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:37.531155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.543524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86361c1-3b78-42e4-bc78-abe4567e47d1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.548949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c0b599-05ef-46d0-8b5e-77fc3a2e7356 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.551677 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.552369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.552724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.552879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.553100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.553458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.553727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.553873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.554008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.555711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.555817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:37.570236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1015/2059] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:37.571313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.586547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9cfb24-0d25-43f2-bb04-fb2bbe68bb1c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.594162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.595711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.595993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.596268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] PATH_INFO: `/v3/projects/948857798c774464a2de4de9b255c56a/users/3e872037e2744e759f38136f6e8b6155/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.603445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:37.612842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948857798c774464a2de4de9b255c56a, user_id=3e872037e2744e759f38136f6e8b6155, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.615753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.649732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1016/2060] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:37 2022] PUT /identity/v3/projects/948857798c774464a2de4de9b255c56a/users/3e872037e2744e759f38136f6e8b6155/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:37.654173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.658706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.660644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.672588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1045/2061] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:37.749430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d882bff-3d38-4958-a079-da6704b7a198 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.765629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54234c77-4f76-4d84-b18e-5a20da12425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.766971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54234c77-4f76-4d84-b18e-5a20da12425e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.767318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54234c77-4f76-4d84-b18e-5a20da12425e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.767657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54234c77-4f76-4d84-b18e-5a20da12425e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.772615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54234c77-4f76-4d84-b18e-5a20da12425e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.772615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54234c77-4f76-4d84-b18e-5a20da12425e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.787675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1017/2062] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:37 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:37.792303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:37.807323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bbb9ea-acf0-4147-a85e-c9340714a5c9 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:37.819394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:37.820871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.821058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.822090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] PATH_INFO: `/v3/projects/948857798c774464a2de4de9b255c56a/users/3e872037e2744e759f38136f6e8b6155/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.827183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=948857798c774464a2de4de9b255c56a, user_id=3e872037e2744e759f38136f6e8b6155, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:37.832206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:37.852124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1046/2063] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:37 2022] PUT /identity/v3/projects/948857798c774464a2de4de9b255c56a/users/3e872037e2744e759f38136f6e8b6155/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:37.863171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:37.863481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:37.863734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:37.896033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] MFA Rules not processed for user `3e872037e2744e759f38136f6e8b6155`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:37.897578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:37.928207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:37.984656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1018/2064] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:37 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:38.033444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:38.094117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c3a87b-7e91-42e7-b4a9-2f2fad959287 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:38.098118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:38.098738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:38.099013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:38.099276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:38.111319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:38.112778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:38.132363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1047/2065] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:38.169692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:38.171791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:38.250306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b7ffeb-e1fa-4f3e-a3c4-019f350b9fb0 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:38.256585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:38.257973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:38.258272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:38.258547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] PATH_INFO: `/v3/users/b193935309c54abda3adf68bf0875fb7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:38.259966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b193935309c54abda3adf68bf0875fb7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:38.263339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:38.283397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:38.293424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f39ac3-5158-4565-bb1e-5ecb8bd3878f service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:38.298237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:38.298717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:38.299008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:38.299860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:38.325787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b193935309c54abda3adf68bf0875fb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:38.326615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b193935309c54abda3adf68bf0875fb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:38.340153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b193935309c54abda3adf68bf0875fb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:38.341106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b193935309c54abda3adf68bf0875fb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:38.359046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-903486db-d0fe-4106-9331-79142265e9a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b193935309c54abda3adf68bf0875fb7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:38.381224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1019/2066] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:38 2022] DELETE /identity/v3/users/b193935309c54abda3adf68bf0875fb7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:38.382240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:38.382930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:38.383463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:38.384323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:38.384816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:38.396815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1048/2067] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:38.408379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:38.410593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:38.455930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:38.487211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1020/2068] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:38.499079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:38.501007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:38.572271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500300a3-dc8f-4f3d-bb2d-ef1cc881c2da service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:38.585985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:38.629367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:38.630350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:38.630808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:38.631256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:38.641715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:38.643388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:38.653850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1049/2069] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:38.967564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:38.969402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:38.982599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758c2bcb-b6d2-4acf-83ed-6e7051ef71be None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:38.988002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:38.988771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:38.989112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:38.989447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] PATH_INFO: `/v3/projects/01f8953a541149969502d6eff6bc8eff` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:38.990653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01f8953a541149969502d6eff6bc8eff)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:38.992719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:39.046229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01f8953a541149969502d6eff6bc8eff', 'request_id': 'req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.052683 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01f8953a541149969502d6eff6bc8eff', 'request_id': 'req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.061070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01f8953a541149969502d6eff6bc8eff', 'request_id': 'req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.074117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] The token cache is being invalidate because project 01f8953a541149969502d6eff6bc8eff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:39.074436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.075076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.076563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1021/2070] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:38 2022] DELETE /identity/v3/projects/01f8953a541149969502d6eff6bc8eff => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:39.080598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:39.090420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2461f9-b432-495b-ae15-36e751396297 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:39.146008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:39.146577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.146849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.147117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] PATH_INFO: `/v3/users/0062ac7c3ff14caaaebea8804725bace` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.148207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0062ac7c3ff14caaaebea8804725bace)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:39.150151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:39.195558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0062ac7c3ff14caaaebea8804725bace'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.196203 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0062ac7c3ff14caaaebea8804725bace'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.203527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0062ac7c3ff14caaaebea8804725bace'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.205013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0062ac7c3ff14caaaebea8804725bace'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.217721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-608c3e49-9c9c-4738-94fe-662606966b5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0062ac7c3ff14caaaebea8804725bace'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.231077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1050/2071] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:39 2022] DELETE /identity/v3/users/0062ac7c3ff14caaaebea8804725bace => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:39.497036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:39.498571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:39.517415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa32c9dd-c4b5-4a88-97cd-54d3a8c20862 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:39.577477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:39.578104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.578414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.578796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] PATH_INFO: `/v3/projects/58c993907d944dc7980c35d419291bc1` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.580060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58c993907d944dc7980c35d419291bc1)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:39.581846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:39.636089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58c993907d944dc7980c35d419291bc1', 'request_id': 'req-5583fbaf-4d2c-4480-9499-0db49d257345'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.642191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58c993907d944dc7980c35d419291bc1', 'request_id': 'req-5583fbaf-4d2c-4480-9499-0db49d257345'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.650039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58c993907d944dc7980c35d419291bc1', 'request_id': 'req-5583fbaf-4d2c-4480-9499-0db49d257345'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.652271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:39.665655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] The token cache is being invalidate because project 58c993907d944dc7980c35d419291bc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:39.666032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.666788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5583fbaf-4d2c-4480-9499-0db49d257345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:39.668334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1022/2072] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:39 2022] DELETE /identity/v3/projects/58c993907d944dc7980c35d419291bc1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:39.673134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.673484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.673817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.695090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:39.696300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:39.730908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:39.760505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1051/2073] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:39.769159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.769444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.769717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.790107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:39.791319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:39.799154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1023/2074] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:39.803511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:39.814314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ec1e17-95ff-480c-a460-3e4bc4992dab None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:39.819168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:39.819775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.820054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.820356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.821008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:39.821311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:39.828194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:39.844936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1052/2075] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:39.846575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:39.855995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46a38f2-653a-40a3-906e-d33d4258e180 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:39.861184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:39.861919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.862191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.862491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.863142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:39.865027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:39.891822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1024/2076] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:39 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:39.897853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:39.908556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b78ef59-11a8-4a05-94d7-f8d8f889c97b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:39.962738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:39.963259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:39.963537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:39.963808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:39.964434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:39.966398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:39.988927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1053/2077] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:39 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:39.993031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.003659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1262e0e-aa60-4544-a790-3189656d8e6c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.008254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.008842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.009123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.009393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.010028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.011643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.022079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1025/2078] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:40.025894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.035603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d27b996-f89a-4bb9-b1a2-a2ab220387d8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.040159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355fa617-668a-40b7-a2d0-fdc7b8829af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.040750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fa617-668a-40b7-a2d0-fdc7b8829af1 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.041024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fa617-668a-40b7-a2d0-fdc7b8829af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.041295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355fa617-668a-40b7-a2d0-fdc7b8829af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.041929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fa617-668a-40b7-a2d0-fdc7b8829af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.043554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355fa617-668a-40b7-a2d0-fdc7b8829af1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.054080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1054/2079] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:40.058218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.068388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bfec8c-ec43-401a-9615-e1cc2f51b001 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.072856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.073487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.073767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.074041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] PATH_INFO: `/v3/projects/aec251ef21974a5fa4d1fb3bcc692a5b/users/95fcd80347b344eb84555321908543d6/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.088931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aec251ef21974a5fa4d1fb3bcc692a5b, user_id=95fcd80347b344eb84555321908543d6, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.091535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.105630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1026/2080] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:40 2022] PUT /identity/v3/projects/aec251ef21974a5fa4d1fb3bcc692a5b/users/95fcd80347b344eb84555321908543d6/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:40.115932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.116417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.116513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.147963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] MFA Rules not processed for user `95fcd80347b344eb84555321908543d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:40.149164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:40.212821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1055/2081] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:40.217385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.278856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb28f4f-af83-451f-8268-9e5cdc034603 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.283528 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.284122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.284400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.284674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.285321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.287230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.313473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1027/2082] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:40 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:40.317617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.328222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab27eb-d6e7-4680-9159-1e6b8b3c8896 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.362849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:40.387949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.388584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.388861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.389156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.389789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.392302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.416211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1056/2083] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:40 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:40.417782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.428171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbf7a3-baa0-4e8a-a11c-119a88f2077a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.432610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.433221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.433496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.433769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.434421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.436022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.447331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1028/2084] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:40.451385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.461872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b89e3-1e22-4144-a73b-e418553d5cd0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.466438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.467032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.467305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.467572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] PATH_INFO: `/v3/projects/7e2d2a9195b84d6ba7ddfe6bff8e623e/users/99eb441fcc404f7a9d3e3619609d9be3/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.482601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2d2a9195b84d6ba7ddfe6bff8e623e, user_id=99eb441fcc404f7a9d3e3619609d9be3, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.485240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.498186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1057/2085] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:40 2022] PUT /identity/v3/projects/7e2d2a9195b84d6ba7ddfe6bff8e623e/users/99eb441fcc404f7a9d3e3619609d9be3/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:40.502445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.512156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f8db4-a9cf-44ae-9933-04ebb1e78302 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.567895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3625767d-2b10-4315-aa5b-9b94f143350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.568495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625767d-2b10-4315-aa5b-9b94f143350b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.568768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625767d-2b10-4315-aa5b-9b94f143350b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.569040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3625767d-2b10-4315-aa5b-9b94f143350b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.569668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625767d-2b10-4315-aa5b-9b94f143350b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.571290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3625767d-2b10-4315-aa5b-9b94f143350b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.581325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1029/2086] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:40 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:40.587877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.598558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8559f3bb-d8fd-4b67-b94e-f4708635701e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.603059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.603689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.603961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.604226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] PATH_INFO: `/v3/projects/7e2d2a9195b84d6ba7ddfe6bff8e623e/users/99eb441fcc404f7a9d3e3619609d9be3/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.606112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e2d2a9195b84d6ba7ddfe6bff8e623e, user_id=99eb441fcc404f7a9d3e3619609d9be3, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.608753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.619327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1058/2087] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:40 2022] PUT /identity/v3/projects/7e2d2a9195b84d6ba7ddfe6bff8e623e/users/99eb441fcc404f7a9d3e3619609d9be3/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:40.631516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.631800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.632073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.639373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:40.661581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] MFA Rules not processed for user `99eb441fcc404f7a9d3e3619609d9be3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:40.662852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:40.746403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1030/2088] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:40 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:40.757282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:40.759139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:40.831229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9da20f-7b59-4dde-af0c-6561e8c38880 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:40.834903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dbeaf0-5a85-47c1-82fd-194799b7f719 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:40.835521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbeaf0-5a85-47c1-82fd-194799b7f719 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:40.835800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbeaf0-5a85-47c1-82fd-194799b7f719 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:40.836073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dbeaf0-5a85-47c1-82fd-194799b7f719 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:40.918224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbeaf0-5a85-47c1-82fd-194799b7f719 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:40.919685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dbeaf0-5a85-47c1-82fd-194799b7f719 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:40.938637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1059/2089] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:41.721611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:41.723099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:41.793507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b05c6f6-a654-47ec-ab49-847b7a2527e5 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:41.798137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:41.798868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:41.799145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:41.799432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] PATH_INFO: `/v3/users/9728d169eee14008a954b521c2e02735` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:41.800539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9728d169eee14008a954b521c2e02735)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:41.802539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:41.853224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9728d169eee14008a954b521c2e02735'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:41.853878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9728d169eee14008a954b521c2e02735'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:41.862150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9728d169eee14008a954b521c2e02735'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:41.862891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9728d169eee14008a954b521c2e02735'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:41.875006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9728d169eee14008a954b521c2e02735'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:41.888214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1031/2090] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:41 2022] DELETE /identity/v3/users/9728d169eee14008a954b521c2e02735 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:41.892812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:41.893357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:41.893357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:41.913271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:41.914504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:41.956182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:41.980901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1060/2091] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:41.997088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:41.998767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:42.012565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:42.083018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e78bb-d782-4334-8dfd-32368315813e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:42.086732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc5be99-d073-4729-b19a-eb7da5aec8a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:42.087571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5be99-d073-4729-b19a-eb7da5aec8a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:42.087952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5be99-d073-4729-b19a-eb7da5aec8a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:42.088335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc5be99-d073-4729-b19a-eb7da5aec8a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:42.102381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5be99-d073-4729-b19a-eb7da5aec8a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:42.103982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc5be99-d073-4729-b19a-eb7da5aec8a0 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:42.119255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1032/2092] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:42.224267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:42.226007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:42.239131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc295297-7b16-4538-9a12-69cae84a5540 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:42.243798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:42.244881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:42.244881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:42.245131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] PATH_INFO: `/v3/users/04b102ea49b74b8e901aea1d9b507cc0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:42.246438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04b102ea49b74b8e901aea1d9b507cc0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:42.248411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:42.302196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04b102ea49b74b8e901aea1d9b507cc0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.303327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04b102ea49b74b8e901aea1d9b507cc0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.313699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04b102ea49b74b8e901aea1d9b507cc0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.314552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04b102ea49b74b8e901aea1d9b507cc0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.330553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04b102ea49b74b8e901aea1d9b507cc0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.351783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1061/2093] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:42 2022] DELETE /identity/v3/users/04b102ea49b74b8e901aea1d9b507cc0 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:42.353415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:42.353696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:42.353967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:42.375253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:42.377742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:42.447140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:42.448971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:42.479496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1033/2094] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:42.490099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:42.495132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:42.555134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd635144-9361-41e0-9c3a-c941f1a89e07 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:42.565426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:42.566362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:42.566966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:42.567433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] PATH_INFO: `/v3/projects/12c1e25efed7429290f625660c601d53` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:42.568929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12c1e25efed7429290f625660c601d53)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:42.571682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:42.596649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d5adbb-632a-4fbd-a8dd-a6c85da5315b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:42.600376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:42.601008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:42.601285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:42.601553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:42.612040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:42.613463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:42.630581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1034/2095] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:42.640717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12c1e25efed7429290f625660c601d53', 'request_id': 'req-9be6ae95-a538-468b-80cc-2a891dcfe9e0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.647847 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12c1e25efed7429290f625660c601d53', 'request_id': 'req-9be6ae95-a538-468b-80cc-2a891dcfe9e0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.650890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:42.659800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12c1e25efed7429290f625660c601d53', 'request_id': 'req-9be6ae95-a538-468b-80cc-2a891dcfe9e0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.676102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] The token cache is being invalidate because project 12c1e25efed7429290f625660c601d53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:42.676529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.677373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.679467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1062/2096] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:42 2022] DELETE /identity/v3/projects/12c1e25efed7429290f625660c601d53 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:42.683758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:42.696904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c27260-77fb-4d88-b94c-512ca5f3f903 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:42.775995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:42.794923 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:42.795758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:42.796611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:42.796611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] PATH_INFO: `/v3/users/70aadf6fd6604066a4a46c829992bf36` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:42.798195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=70aadf6fd6604066a4a46c829992bf36)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:42.802832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:42.866874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70aadf6fd6604066a4a46c829992bf36'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.867797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70aadf6fd6604066a4a46c829992bf36'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.875544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70aadf6fd6604066a4a46c829992bf36'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.876374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70aadf6fd6604066a4a46c829992bf36'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.893809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70aadf6fd6604066a4a46c829992bf36'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:42.910626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1035/2097] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:42 2022] DELETE /identity/v3/users/70aadf6fd6604066a4a46c829992bf36 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:42.970951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:42.972573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:42.994676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be6ae95-a538-468b-80cc-2a891dcfe9e0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.094349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.094349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.094349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.094349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] PATH_INFO: `/v3/projects/8e031c5a09954ec986fd00aab53512ab` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.095165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e031c5a09954ec986fd00aab53512ab)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.095165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.181552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e031c5a09954ec986fd00aab53512ab', 'request_id': 'req-62acac18-c0b3-441e-9701-83c6c115332b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.189480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e031c5a09954ec986fd00aab53512ab', 'request_id': 'req-62acac18-c0b3-441e-9701-83c6c115332b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.199824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e031c5a09954ec986fd00aab53512ab', 'request_id': 'req-62acac18-c0b3-441e-9701-83c6c115332b'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.220249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] The token cache is being invalidate because project 8e031c5a09954ec986fd00aab53512ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:43.221037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.221803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-62acac18-c0b3-441e-9701-83c6c115332b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.223931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1063/2098] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:42 2022] DELETE /identity/v3/projects/8e031c5a09954ec986fd00aab53512ab => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:43.293504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.295555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:43.313713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483eb1b4-e43a-4f84-b9f2-5ff2b3cc01fe admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.432842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.433476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.433750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.434026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] PATH_INFO: `/v3/projects/2b8cd6efab3f435ca138c3d7636eae01` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.436946 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2b8cd6efab3f435ca138c3d7636eae01)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.438558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.485356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2b8cd6efab3f435ca138c3d7636eae01', 'request_id': 'req-ab87f173-c35f-4b80-839c-7df05a466290'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.491053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2b8cd6efab3f435ca138c3d7636eae01', 'request_id': 'req-ab87f173-c35f-4b80-839c-7df05a466290'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.498961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2b8cd6efab3f435ca138c3d7636eae01', 'request_id': 'req-ab87f173-c35f-4b80-839c-7df05a466290'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.511473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] The token cache is being invalidate because project 2b8cd6efab3f435ca138c3d7636eae01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:43.511758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.512431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ab87f173-c35f-4b80-839c-7df05a466290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:43.513912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1036/2099] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:43 2022] DELETE /identity/v3/projects/2b8cd6efab3f435ca138c3d7636eae01 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:43.518863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.519153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.519432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.539813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:43.541020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:43.608678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1064/2100] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:43.617635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.617992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.618288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.628640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:43.647665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:43.648959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:43.657511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1037/2101] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:43.668158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.674339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205042dd-9d19-4c01-9068-f3871197be8b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.679466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.680133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.680412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.680680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.681354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:43.681654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.683327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.700498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1065/2102] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:43.701745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.711176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cc7d2b-26bd-4910-a2f1-f97f3f6635d8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.715956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.716617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.716963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.717297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.718021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.723014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.755461 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1038/2103] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:43 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:43.760241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.765534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:43.773335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb8f04e-e7cc-401a-a05e-2d77c91e46a0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.859222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.860379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.860679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.860924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.861554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.863513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.888396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1066/2104] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:43 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:43.894230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.905418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9111d4d1-320f-4836-9e1f-b26b3ab40a23 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.911412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.912248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.912884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.913160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.913815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.915397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.934457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1039/2105] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:43.934575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.941025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36571ea1-8024-4ffe-953c-c20441005f98 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.945879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e447151-0498-4016-95d2-bb98b69be15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.946594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447151-0498-4016-95d2-bb98b69be15a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.946955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447151-0498-4016-95d2-bb98b69be15a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.947215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e447151-0498-4016-95d2-bb98b69be15a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:43.947853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e447151-0498-4016-95d2-bb98b69be15a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:43.949445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e447151-0498-4016-95d2-bb98b69be15a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:43.962679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1067/2106] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:43 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:43.966968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:43.978584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d21a75-483c-41f7-9724-447b8a906c77 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:43.983573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:43.984219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:43.984503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:43.984816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] PATH_INFO: `/v3/projects/6198681799504a99b47b9f65a5d6212f/users/1293f043ff794425b07aa772c8460c77/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.004507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6198681799504a99b47b9f65a5d6212f, user_id=1293f043ff794425b07aa772c8460c77, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.007177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.020931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1040/2107] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:43 2022] PUT /identity/v3/projects/6198681799504a99b47b9f65a5d6212f/users/1293f043ff794425b07aa772c8460c77/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:44.032639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.033103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.033191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.061527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] MFA Rules not processed for user `1293f043ff794425b07aa772c8460c77`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:44.062769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.130681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1068/2108] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:44.135443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.221314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53838c64-e9e5-49c6-be7d-fcb3093597d2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.227012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.227955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.228160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.228565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.229216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.231793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.267645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1041/2109] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:44 2022] POST /identity/v3/projects => generated 381 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:44.272821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.279995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bae486-770f-4920-a4ab-5bf075d88eb1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.338222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.338855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.339134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.339407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.340049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.341968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.358570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:44.367097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1069/2110] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:44 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:44.368275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.379143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120e4de5-c0a6-49ff-8694-538dc28e874c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.383735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.384336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.384613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.384885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.385520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.387134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.402017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1042/2111] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:44.406329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.407318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.409006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.413550 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64084874-73dc-44cb-9ef4-52a17b434694 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.418423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.419098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.419372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.419644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] PATH_INFO: `/v3/projects/f145a4debf5f4151b2c7b29aba293a99/users/742ba217d9454ab5a56db991475c7a19/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.423794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce539d0-7a0a-4b69-8535-f47292b618b9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.429081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.429717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.429997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.430318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] PATH_INFO: `/v3/users/e67a5ab134894180a7f3a6943b5b7991` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.431608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e67a5ab134894180a7f3a6943b5b7991)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.433520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.437403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f145a4debf5f4151b2c7b29aba293a99, user_id=742ba217d9454ab5a56db991475c7a19, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.440736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.456549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1070/2112] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:44 2022] PUT /identity/v3/projects/f145a4debf5f4151b2c7b29aba293a99/users/742ba217d9454ab5a56db991475c7a19/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:44.462422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.487543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e67a5ab134894180a7f3a6943b5b7991'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:44.488529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e67a5ab134894180a7f3a6943b5b7991'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:44.497240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e67a5ab134894180a7f3a6943b5b7991'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:44.498169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e67a5ab134894180a7f3a6943b5b7991'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:44.508019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:44.514188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3c817f4d-1872-450b-9f25-fe97aa6cca74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e67a5ab134894180a7f3a6943b5b7991'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:44.536406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef718f-5125-4f64-aa00-a44c4c1f489f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.542179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1043/2113] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:44 2022] DELETE /identity/v3/users/e67a5ab134894180a7f3a6943b5b7991 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:44.542376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.542871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.543147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.543472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.543902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.544253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.544343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.544760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.546030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.558288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1071/2114] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:44.562329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.564283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.566273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:44.567210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.577534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99866dce-0098-4141-9e9a-c5c5045b4230 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.640469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1044/2115] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:44.648075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30cd97-0913-499f-8f2f-fd42a0b4a1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.648736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cd97-0913-499f-8f2f-fd42a0b4a1b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.649011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cd97-0913-499f-8f2f-fd42a0b4a1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.649281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30cd97-0913-499f-8f2f-fd42a0b4a1b3 admin admin] PATH_INFO: `/v3/projects/f145a4debf5f4151b2c7b29aba293a99/users/742ba217d9454ab5a56db991475c7a19/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.650777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.651301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30cd97-0913-499f-8f2f-fd42a0b4a1b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f145a4debf5f4151b2c7b29aba293a99, user_id=742ba217d9454ab5a56db991475c7a19, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.652150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.654089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30cd97-0913-499f-8f2f-fd42a0b4a1b3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.666417 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1072/2116] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:44 2022] PUT /identity/v3/projects/f145a4debf5f4151b2c7b29aba293a99/users/742ba217d9454ab5a56db991475c7a19/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:44.677168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.677447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.677716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.705718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] MFA Rules not processed for user `742ba217d9454ab5a56db991475c7a19`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:44.707063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.709944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762858af-6794-4221-8463-ecc1f7f89f5d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.757520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:44.759773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.760382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.760654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.760922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.776834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1073/2117] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:44 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:44.788243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:44.789598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:44.812215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:44.822962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.824832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.834341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1045/2118] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:44.851583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b5b67f-a399-476a-bc58-887f1874a7ac None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:44.856731 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:44.857501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:44.857762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:44.858085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:44.872551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:44.874100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:44.884666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1074/2119] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:44 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:45.126709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:45.128135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:45.142348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d0bd4-ad31-4fc4-9339-8f31cbdb2974 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:45.152038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:45.153441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.154759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.155202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] PATH_INFO: `/v3/projects/bc2b312831c8475caf68f2e9a9fe6e24` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.156515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc2b312831c8475caf68f2e9a9fe6e24)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:45.159543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:45.222897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc2b312831c8475caf68f2e9a9fe6e24', 'request_id': 'req-331d4e2f-fbc9-4db6-bd03-27da016841ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.230635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc2b312831c8475caf68f2e9a9fe6e24', 'request_id': 'req-331d4e2f-fbc9-4db6-bd03-27da016841ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.239646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc2b312831c8475caf68f2e9a9fe6e24', 'request_id': 'req-331d4e2f-fbc9-4db6-bd03-27da016841ee'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.252370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] The token cache is being invalidate because project bc2b312831c8475caf68f2e9a9fe6e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:45.252775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.253613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.268146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1046/2120] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:45 2022] DELETE /identity/v3/projects/bc2b312831c8475caf68f2e9a9fe6e24 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:45.271891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:45.283609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced815b0-d07c-4198-8ebb-685b297d9af9 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:45.327480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:45.330181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:45.351187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:45.351826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.352104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.352376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] PATH_INFO: `/v3/users/3e872037e2744e759f38136f6e8b6155` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.353498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e872037e2744e759f38136f6e8b6155)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:45.355496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:45.396525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331d4e2f-fbc9-4db6-bd03-27da016841ee admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:45.401238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:45.401968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.402077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e872037e2744e759f38136f6e8b6155'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.402465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.402693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e872037e2744e759f38136f6e8b6155'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.402805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] PATH_INFO: `/v3/users/1293f043ff794425b07aa772c8460c77` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.403850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1293f043ff794425b07aa772c8460c77)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:45.405745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:45.410220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e872037e2744e759f38136f6e8b6155'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.411900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e872037e2744e759f38136f6e8b6155'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.424529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4b4f50f1-c215-47a8-8cbd-79ce5aa79ba6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e872037e2744e759f38136f6e8b6155'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.437637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1075/2121] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:45 2022] DELETE /identity/v3/users/3e872037e2744e759f38136f6e8b6155 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:45.455517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1293f043ff794425b07aa772c8460c77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.456402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1293f043ff794425b07aa772c8460c77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.463380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1293f043ff794425b07aa772c8460c77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.464038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1293f043ff794425b07aa772c8460c77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.477189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1293f043ff794425b07aa772c8460c77'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.480199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:45.493060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1047/2122] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:45 2022] DELETE /identity/v3/users/1293f043ff794425b07aa772c8460c77 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:45.497564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.497884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.498158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.519795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:45.521086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:45.529906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:45.582602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1076/2123] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:45.592737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:45.595119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:45.665804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246bb947-4958-41ff-9bd2-1a66a226e6f4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:45.669717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9134a13d-80ef-4387-b8ab-db160e8fa401 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:45.670534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134a13d-80ef-4387-b8ab-db160e8fa401 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.670879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134a13d-80ef-4387-b8ab-db160e8fa401 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.671080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9134a13d-80ef-4387-b8ab-db160e8fa401 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.682594 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134a13d-80ef-4387-b8ab-db160e8fa401 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:45.684032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9134a13d-80ef-4387-b8ab-db160e8fa401 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:45.694947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1048/2124] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:45.758869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:45.763315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:45.783138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58069972-dc5d-4899-a6cb-21015c6e3225 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:45.789351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:45.790134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.790542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.790945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] PATH_INFO: `/v3/projects/948857798c774464a2de4de9b255c56a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.792252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=948857798c774464a2de4de9b255c56a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:45.795852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:45.850420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '948857798c774464a2de4de9b255c56a', 'request_id': 'req-0f2789f7-0a9f-481d-8532-ed89081fb0c6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.856919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '948857798c774464a2de4de9b255c56a', 'request_id': 'req-0f2789f7-0a9f-481d-8532-ed89081fb0c6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.863238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '948857798c774464a2de4de9b255c56a', 'request_id': 'req-0f2789f7-0a9f-481d-8532-ed89081fb0c6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.879524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] The token cache is being invalidate because project 948857798c774464a2de4de9b255c56a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:45.879923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.886839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0f2789f7-0a9f-481d-8532-ed89081fb0c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:45.886839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1077/2125] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:45 2022] DELETE /identity/v3/projects/948857798c774464a2de4de9b255c56a => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:45.889042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.889472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.889737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:45.913100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:45.914367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:45.981287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1049/2126] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:45.989400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:45.989771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:45.990111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.010899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:46.012092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.017728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.019137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.023518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1078/2127] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:46.027496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.032441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5d7105-7024-4ed6-8cdb-c8076cc5009a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.037965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.038636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.039082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.039333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] PATH_INFO: `/v3/projects/6198681799504a99b47b9f65a5d6212f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.040522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6198681799504a99b47b9f65a5d6212f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.042761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.050067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c20e6d1-5eba-464d-aca8-99ecec254906 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.057291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.057936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.058277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.058553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.059219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:46.059506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.062486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.078313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1079/2128] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:21:46.084576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.095901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64187b-cee0-41e9-8e7a-6cfbaacec817 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.101269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6198681799504a99b47b9f65a5d6212f', 'request_id': 'req-6900d6eb-6312-46ac-8fd8-3308f133bfb1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.107464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6198681799504a99b47b9f65a5d6212f', 'request_id': 'req-6900d6eb-6312-46ac-8fd8-3308f133bfb1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.115563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6198681799504a99b47b9f65a5d6212f', 'request_id': 'req-6900d6eb-6312-46ac-8fd8-3308f133bfb1'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.127749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] The token cache is being invalidate because project 6198681799504a99b47b9f65a5d6212f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:46.128372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.128679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.130179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1050/2129] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:46 2022] DELETE /identity/v3/projects/6198681799504a99b47b9f65a5d6212f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:46.134087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.135399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.157949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:46.159869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.160507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.160786 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.161056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.161682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.163639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.191644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1080/2130] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/projects => generated 383 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:46.195874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.197552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.201798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:46.211826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6900d6eb-6312-46ac-8fd8-3308f133bfb1 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.267868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebce1c1-1cc7-47bb-97d7-346053f1018b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.273193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.273807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.274106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.274407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.275040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.276963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.280019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.280933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.281345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.282692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] PATH_INFO: `/v3/users/742ba217d9454ab5a56db991475c7a19` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.283957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=742ba217d9454ab5a56db991475c7a19)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.287269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.304903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1081/2131] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/users => generated 292 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:46.308823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.330352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8e01a5-2547-4613-8ebf-86775c9dec48 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.336527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.337157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.337433 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.337709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.338394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.340076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.353379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '742ba217d9454ab5a56db991475c7a19'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.354045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '742ba217d9454ab5a56db991475c7a19'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.356618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1082/2132] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:46.361488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.362004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '742ba217d9454ab5a56db991475c7a19'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.362838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '742ba217d9454ab5a56db991475c7a19'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.372465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d47e60b-47eb-4a1b-8a5e-ba7db9d19fed admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.376414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-62458504-0987-45e8-b6ed-47bacf55cca4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '742ba217d9454ab5a56db991475c7a19'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.377180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.377784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.378062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.378389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.379032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.380647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.395397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1083/2133] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:46.396412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.398739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.398960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1051/2134] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:46 2022] DELETE /identity/v3/users/742ba217d9454ab5a56db991475c7a19 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:46.412755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f396643d-7acd-4ad1-9838-66d8ec5a63e6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.472180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.472852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.473131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.473410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] PATH_INFO: `/v3/projects/0dbb4760a75a4c4c9248c84c06d700b9/users/16030628dd99424c934a872e3cd8735c/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.490329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dbb4760a75a4c4c9248c84c06d700b9, user_id=16030628dd99424c934a872e3cd8735c, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.493502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.507038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1084/2135] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:46 2022] PUT /identity/v3/projects/0dbb4760a75a4c4c9248c84c06d700b9/users/16030628dd99424c934a872e3cd8735c/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:46.518813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.519095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.519368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.549879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] MFA Rules not processed for user `16030628dd99424c934a872e3cd8735c`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:46.551122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.588201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:46.626445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1052/2136] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:46.631033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.683253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.684661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.692187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:46.710648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18570a-fff0-4173-808f-9e730792c6e6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.717939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.718579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.719704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.720100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.720809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.722749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.751262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1085/2137] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:46.755235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.762218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5654d581-741b-4b41-b44a-c96ff32fdb7c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.821826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.821826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.821826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.822141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac13c6a-1791-4ef5-ae0f-580c2231d3e0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.822409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] PATH_INFO: `/v3/projects/f145a4debf5f4151b2c7b29aba293a99` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.823926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f145a4debf5f4151b2c7b29aba293a99)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.825860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.826989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.827593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.827869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.828138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.829328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.831427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.854893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1086/2138] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:46.858549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.868605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4942e-4adf-4e9a-bada-340d8812b4b1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.882548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f145a4debf5f4151b2c7b29aba293a99', 'request_id': 'req-99aac42d-b98a-4604-ab9c-782b3d050de6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.888909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f145a4debf5f4151b2c7b29aba293a99', 'request_id': 'req-99aac42d-b98a-4604-ab9c-782b3d050de6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.899606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f145a4debf5f4151b2c7b29aba293a99', 'request_id': 'req-99aac42d-b98a-4604-ab9c-782b3d050de6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.911938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] The token cache is being invalidate because project f145a4debf5f4151b2c7b29aba293a99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:46.912290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.913008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-99aac42d-b98a-4604-ab9c-782b3d050de6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:46.914636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1053/2139] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:46 2022] DELETE /identity/v3/projects/f145a4debf5f4151b2c7b29aba293a99 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:46.920282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d561deb-ba10-4c5e-b770-9f99335b5f13 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.920720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d561deb-ba10-4c5e-b770-9f99335b5f13 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.920871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d561deb-ba10-4c5e-b770-9f99335b5f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.935717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.936646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.937060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.937407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.938822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:46.940789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:46.944002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-0d561deb-ba10-4c5e-b770-9f99335b5f13 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:46.945219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0d561deb-ba10-4c5e-b770-9f99335b5f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.956851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1087/2140] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:46.962420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:46.962566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1054/2141] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:46.964721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:46.971387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.971670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.971945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.978020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da6f78-67bf-412a-aa99-a89fd2ca1bb1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:46.982706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:46.983486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:46.983766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:46.984042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] PATH_INFO: `/v3/projects/ea9922838ba04f0f974f329c9f233d19/users/9e5ad687a2574ce0bbda841b4644b1af/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:46.987166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:46.994942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:46.996851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.007911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9922838ba04f0f974f329c9f233d19, user_id=9e5ad687a2574ce0bbda841b4644b1af, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.007911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.008205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1055/2142] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:47.013724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.023980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2a37a-ff0e-46e1-98a4-351518faf304 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.024218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1088/2143] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:46 2022] PUT /identity/v3/projects/ea9922838ba04f0f974f329c9f233d19/users/9e5ad687a2574ce0bbda841b4644b1af/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:47.028450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.076024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:47.083813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.084426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.084732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.085104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.085647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:47.085944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.087574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.097825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b75eba-fb08-45b0-8b48-ae3040db9ee0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.100406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1056/2144] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:21:47.102729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.103441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.103697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.103964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.104596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.104701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.106174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.114854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ba754a-045f-48d9-8ad6-20a22380f82d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.117826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1089/2145] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:47.119803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.120476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.120831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.121109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.121765 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.121864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.123723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.132477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac55b2-5c2d-4db9-a9e1-a2210a026fe5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.137072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921f2ea-f329-40da-a0c3-83b3d332f6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.137701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f2ea-f329-40da-a0c3-83b3d332f6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.137974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f2ea-f329-40da-a0c3-83b3d332f6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.138268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921f2ea-f329-40da-a0c3-83b3d332f6cd admin admin] PATH_INFO: `/v3/projects/ea9922838ba04f0f974f329c9f233d19/users/9e5ad687a2574ce0bbda841b4644b1af/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.140146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921f2ea-f329-40da-a0c3-83b3d332f6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea9922838ba04f0f974f329c9f233d19, user_id=9e5ad687a2574ce0bbda841b4644b1af, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.143554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8921f2ea-f329-40da-a0c3-83b3d332f6cd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.154689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1057/2146] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:47 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:47.155610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.158546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1090/2147] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:47 2022] PUT /identity/v3/projects/ea9922838ba04f0f974f329c9f233d19/users/9e5ad687a2574ce0bbda841b4644b1af/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:47.168488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.168775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.169051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.198968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] MFA Rules not processed for user `9e5ad687a2574ce0bbda841b4644b1af`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:47.200127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.220507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7d1dc9-0008-41f0-8bce-91cd60f9a69a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.225098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.225700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.225978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.226282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.226909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.228810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.253566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1058/2148] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:47 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:47.259241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.270142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399948ab-6947-44dd-b10e-5fd2a61bd660 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.270533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1091/2149] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:47.274814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.275412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.275694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.275968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.276606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.278263 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.283149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.284526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.292841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1059/2150] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:47.293931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.305019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028df82-2c6b-4ffa-9ac1-b63c4318ddc5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.309509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.310088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.310394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.310665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.311283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.312897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.323550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1060/2151] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:47.327371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.337695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759b4ca1-36e4-4a83-ae39-a5f3e00c0dde admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.343689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.344383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302634c-b144-4b69-b1c1-dfe70f4ac258 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.345015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.345364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.345699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] PATH_INFO: `/v3/projects/8974faa3ab6f4683b0a502a2477dfbb6/users/c77e17d0ee6d4e00bbbe7c988c4fad6d/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.348290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bfcb0c-8b3f-4250-a984-16d7286a69b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.348888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcb0c-8b3f-4250-a984-16d7286a69b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.349964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcb0c-8b3f-4250-a984-16d7286a69b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.349964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bfcb0c-8b3f-4250-a984-16d7286a69b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.363469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcb0c-8b3f-4250-a984-16d7286a69b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.363858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8974faa3ab6f4683b0a502a2477dfbb6, user_id=c77e17d0ee6d4e00bbbe7c988c4fad6d, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.364904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bfcb0c-8b3f-4250-a984-16d7286a69b5 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.366530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.377830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1092/2152] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:47.397454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1061/2153] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:47 2022] PUT /identity/v3/projects/8974faa3ab6f4683b0a502a2477dfbb6/users/c77e17d0ee6d4e00bbbe7c988c4fad6d/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:47.403955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.404275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.404576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.465725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] MFA Rules not processed for user `c77e17d0ee6d4e00bbbe7c988c4fad6d`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:47.466958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.479186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.481587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.505644 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:47.528655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:47.555761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77fbc8-d579-4ed1-b8d1-67fdc3570307 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.559368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.559970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.560246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.560512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.562079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1093/2154] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:47.562288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.576084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.589922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024f3548-c681-49da-a4e7-41311c00ed59 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.597285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.597907 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.598180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.598481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.607956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.609392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.618592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1094/2155] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:47.639216 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.652762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.654406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c89a4c-25e1-49b7-86ae-b01aaab2f56d service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.663214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1062/2156] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:47.707067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163a1b54-10ba-46cc-84ef-5cc9ebadfb49 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.711615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.712241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.712621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.712897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] PATH_INFO: `/v3/users/c77e17d0ee6d4e00bbbe7c988c4fad6d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.714152 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=c77e17d0ee6d4e00bbbe7c988c4fad6d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.716136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.773430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c77e17d0ee6d4e00bbbe7c988c4fad6d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:47.774272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c77e17d0ee6d4e00bbbe7c988c4fad6d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:47.781466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c77e17d0ee6d4e00bbbe7c988c4fad6d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:47.782142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c77e17d0ee6d4e00bbbe7c988c4fad6d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:47.795869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c77e17d0ee6d4e00bbbe7c988c4fad6d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:47.808890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1095/2157] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:47 2022] DELETE /identity/v3/users/c77e17d0ee6d4e00bbbe7c988c4fad6d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:47.814073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.814466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.814750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.834004 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:47.835262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.901605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1063/2158] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:47.911720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.913169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.956852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:47.958524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:47.974000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2f336a-48f6-42b3-a0c2-4b0fc320b707 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.979776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.980520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.981020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.981238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:47.981493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] PATH_INFO: `/v3/users/16030628dd99424c934a872e3cd8735c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:47.982762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] RBAC: Authorizing `identity:delete_user(user_id=16030628dd99424c934a872e3cd8735c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:47.986465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:47.991570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b45125c-be50-4ba3-ba72-95e9a58e96ab admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:47.997559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108edeec-cb51-45f4-aeca-e7b8651d9a38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:47.998052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108edeec-cb51-45f4-aeca-e7b8651d9a38 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:47.998403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108edeec-cb51-45f4-aeca-e7b8651d9a38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:47.998658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108edeec-cb51-45f4-aeca-e7b8651d9a38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.009177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108edeec-cb51-45f4-aeca-e7b8651d9a38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:48.010801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108edeec-cb51-45f4-aeca-e7b8651d9a38 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:48.023602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1096/2159] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:48.052118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16030628dd99424c934a872e3cd8735c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.053237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16030628dd99424c934a872e3cd8735c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.062308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16030628dd99424c934a872e3cd8735c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.063124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16030628dd99424c934a872e3cd8735c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.074479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:48.083838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16030628dd99424c934a872e3cd8735c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.103729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1064/2160] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:47 2022] DELETE /identity/v3/users/16030628dd99424c934a872e3cd8735c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:48.108270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.108554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.108818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.131896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:48.133200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.209343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1097/2161] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:48.220127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:48.221622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.308691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f14898-ddf8-4218-a287-6bc4a77cf37a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:48.312401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:48.313950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.313950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.314131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.326756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:48.328234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:48.342216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1065/2162] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:48.500803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:48.502789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.516923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70a37d3-b27f-4925-9a4c-2809998e347c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:48.535751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:48.535751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.535751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.535751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] PATH_INFO: `/v3/projects/8974faa3ab6f4683b0a502a2477dfbb6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.536692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8974faa3ab6f4683b0a502a2477dfbb6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:48.536692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:48.613194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:48.614714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.635646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8974faa3ab6f4683b0a502a2477dfbb6', 'request_id': 'req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.642556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8974faa3ab6f4683b0a502a2477dfbb6', 'request_id': 'req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.651420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8974faa3ab6f4683b0a502a2477dfbb6', 'request_id': 'req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.671729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] The token cache is being invalidate because project 8974faa3ab6f4683b0a502a2477dfbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:48.672259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.673519 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5c5b2d05-1a3a-46d0-9738-dd522b1b4d81 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.675892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1098/2163] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:48 2022] DELETE /identity/v3/projects/8974faa3ab6f4683b0a502a2477dfbb6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:48.680556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.681262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.681801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.717491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:48.720646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.739847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cda442-ced2-4a08-9c5a-6d4b5bc13431 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:48.815244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:48.815859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.816142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.816415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.818024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1099/2164] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:48.818475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.819147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:48.820537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.833077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc784a39-21ad-4d28-a7d6-dc36eccb6c0d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:48.837777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:48.838440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.838719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.838991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] PATH_INFO: `/v3/projects/0dbb4760a75a4c4c9248c84c06d700b9` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.840107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dbb4760a75a4c4c9248c84c06d700b9)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:48.842027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:48.851921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:48.885904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:48.903013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:48.903289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dbb4760a75a4c4c9248c84c06d700b9', 'request_id': 'req-26baae1a-531e-40e9-8673-ea4693ef8ed6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.905195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7da7c7-0667-4530-91ac-6bfa026e780c service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:48.910796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dbb4760a75a4c4c9248c84c06d700b9', 'request_id': 'req-26baae1a-531e-40e9-8673-ea4693ef8ed6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.917075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dbb4760a75a4c4c9248c84c06d700b9', 'request_id': 'req-26baae1a-531e-40e9-8673-ea4693ef8ed6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.917972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1066/2165] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:48.919368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:48.919709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:48.920054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:48.934911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] The token cache is being invalidate because project 0dbb4760a75a4c4c9248c84c06d700b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:48.935211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.935901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:48.940913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1100/2166] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:48 2022] DELETE /identity/v3/projects/0dbb4760a75a4c4c9248c84c06d700b9 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:48.945441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:48.946618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:48.946737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:48.948262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.025626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1067/2167] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:49.029672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.033257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baae1a-531e-40e9-8673-ea4693ef8ed6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.038141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.038809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.039088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.039369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] PATH_INFO: `/v3/users/9e5ad687a2574ce0bbda841b4644b1af` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.040463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e5ad687a2574ce0bbda841b4644b1af)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.040908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f3a480-4fd0-43f4-a6f8-ecef823312cd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.042402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.045651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.046266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.046550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.046830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.047638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:49.047942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.049529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.068733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1068/2168] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:21:49.069799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.081112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4c4892-64b2-4b00-97eb-97b36af8a3ba admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.086317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.087113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.087397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.087895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.088669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.090914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.098535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e5ad687a2574ce0bbda841b4644b1af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.099386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e5ad687a2574ce0bbda841b4644b1af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.108015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e5ad687a2574ce0bbda841b4644b1af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.108707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e5ad687a2574ce0bbda841b4644b1af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.123399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1069/2169] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/projects => generated 361 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:49.124982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e5ad687a2574ce0bbda841b4644b1af'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.128617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.130408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.143089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1101/2170] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:48 2022] DELETE /identity/v3/users/9e5ad687a2574ce0bbda841b4644b1af => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:49.235187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d62793-3612-4d7e-8fc3-e52bb302261d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.331520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.334097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.334097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.334340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.334862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.337466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.352473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:49.377915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1070/2171] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/users => generated 281 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:49.378876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.396158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5d5db5-ce3d-44eb-9f6d-8630a2cb132a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.410197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.412233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.412523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.412802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.413475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.415122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.444665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1102/2172] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:49.448647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.462361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db3f483-4a2f-47ce-af57-16d256084e08 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.472754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.473414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.473720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.474173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.474753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.477475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.519462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1071/2173] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:49.521150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.531162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:49.538511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.543348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.563391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff797272-a1d6-47be-8143-10f25028a4ef admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.572088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e0f289-7769-4960-8c05-e7818b5323ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.572768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f289-7769-4960-8c05-e7818b5323ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.573043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f289-7769-4960-8c05-e7818b5323ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.573312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e0f289-7769-4960-8c05-e7818b5323ca admin admin] PATH_INFO: `/v3/projects/06bae63a727d42f1a7692887066f9d6e/users/43375ff9777543e997e6280da86d7599/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.575905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a496ce68-8dc4-458f-84ba-8c64abccb728 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.581967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.582652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.582935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.583208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] PATH_INFO: `/v3/projects/ea9922838ba04f0f974f329c9f233d19` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.585573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea9922838ba04f0f974f329c9f233d19)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.587510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.599619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0f289-7769-4960-8c05-e7818b5323ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=06bae63a727d42f1a7692887066f9d6e, user_id=43375ff9777543e997e6280da86d7599, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.604977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e0f289-7769-4960-8c05-e7818b5323ca admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.622367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1103/2174] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:49 2022] PUT /identity/v3/projects/06bae63a727d42f1a7692887066f9d6e/users/43375ff9777543e997e6280da86d7599/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:49.633648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.634061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.634694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.659939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea9922838ba04f0f974f329c9f233d19', 'request_id': 'req-aef793ac-f334-43e3-af45-1d66687204bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.665665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea9922838ba04f0f974f329c9f233d19', 'request_id': 'req-aef793ac-f334-43e3-af45-1d66687204bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.666982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] MFA Rules not processed for user `43375ff9777543e997e6280da86d7599`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:49.668205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.674516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea9922838ba04f0f974f329c9f233d19', 'request_id': 'req-aef793ac-f334-43e3-af45-1d66687204bf'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.686232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] The token cache is being invalidate because project ea9922838ba04f0f974f329c9f233d19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:49.686520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.687543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-aef793ac-f334-43e3-af45-1d66687204bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.689061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1072/2175] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:49 2022] DELETE /identity/v3/projects/ea9922838ba04f0f974f329c9f233d19 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:49.693922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91aaa-01b7-490e-8b5e-1215a8b997ca None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.694216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91aaa-01b7-490e-8b5e-1215a8b997ca None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.694518 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c91aaa-01b7-490e-8b5e-1215a8b997ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.715729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-47c91aaa-01b7-490e-8b5e-1215a8b997ca None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:49.716988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-47c91aaa-01b7-490e-8b5e-1215a8b997ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.735964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1104/2176] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:49.772826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.774233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.781107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1073/2177] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:49.789319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.789922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.790401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.811168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:49.812376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.821222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1074/2178] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:49.825365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.833758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e6bab-6393-421f-83cf-0c193fd2af82 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.837012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39cbe93-86f3-4835-b08f-f9bf5183f589 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.837466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.838071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.838373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.838637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.841830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.842444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.842727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.843000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.844736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:49.845049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.846691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.847863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.849285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.858318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1105/2179] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:49 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:49.862741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1075/2180] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:21:49.863769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.874541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.876286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:49.876941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e87b29-9221-46d2-9b8a-3749266f4114 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.881459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.882038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.882339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.882717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.883340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.885524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.888807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a74dea-723f-4558-8796-ff1359fdd7a6 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:49.893881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:49.894542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:49.894823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:49.895097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] PATH_INFO: `/v3/users/43375ff9777543e997e6280da86d7599` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:49.896251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] RBAC: Authorizing `identity:delete_user(user_id=43375ff9777543e997e6280da86d7599)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:49.898141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:49.916477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1106/2181] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:49.917477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:49.946468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=2,keystone:SELECT=85,keystone:DELETE=13 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:49.952965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43375ff9777543e997e6280da86d7599'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.955869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43375ff9777543e997e6280da86d7599'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.964839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43375ff9777543e997e6280da86d7599'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.965576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43375ff9777543e997e6280da86d7599'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.979202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:49.986620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f320ff1-4030-4ca7-afee-1bcb235d00be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43375ff9777543e997e6280da86d7599'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:49.997006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2311223-51cf-43bb-ace7-6e9452ef1f27 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.005531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.005531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.006022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.006377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.007009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.008220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1076/2182] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:49 2022] DELETE /identity/v3/users/43375ff9777543e997e6280da86d7599 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:50.010153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.012283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.012636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.012975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.035573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:50.036793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.039228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1107/2183] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:49 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:21:50.043038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.044724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.057341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce106e53-1cd7-4699-9e9c-d4cb28a3b877 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.102480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1077/2184] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:50.112535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.113927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.127327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.127939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.128218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.128489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.129136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.131459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.143426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1108/2185] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:50.148140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.173532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe358b8c-0bfe-41f5-b8f9-a72ca5a17b4c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.176284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4209086-ee2a-42c2-8fb9-735a22b80727 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.178529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.179126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.179404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.179677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.179865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dd1c9-8669-40a0-b28d-c14642fd6137 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.180444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.180538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dd1c9-8669-40a0-b28d-c14642fd6137 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.180716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dd1c9-8669-40a0-b28d-c14642fd6137 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.180986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dd1c9-8669-40a0-b28d-c14642fd6137 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.182041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.190465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dd1c9-8669-40a0-b28d-c14642fd6137 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.191871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dd1c9-8669-40a0-b28d-c14642fd6137 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.193175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1109/2186] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:50.197183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.201434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1078/2187] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:50.207832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ba82bc-79c6-4f56-9326-3e625edc8c72 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.212418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.213064 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.213342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.213613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] PATH_INFO: `/v3/projects/df77f2396aae4df6a5634bf18d492fca/users/1a4d8e5164f344e384c2a525fba2caa5/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.229756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] RBAC: Authorizing `identity:create_grant(project_id=df77f2396aae4df6a5634bf18d492fca, user_id=1a4d8e5164f344e384c2a525fba2caa5, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.232446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.246165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1110/2188] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:50 2022] PUT /identity/v3/projects/df77f2396aae4df6a5634bf18d492fca/users/1a4d8e5164f344e384c2a525fba2caa5/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:50.260301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.260796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.261023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.291625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] MFA Rules not processed for user `1a4d8e5164f344e384c2a525fba2caa5`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:50.292829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.367502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:50.393459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1079/2189] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:50.405947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.408818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.489463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff623-e1e4-4895-ac5c-095ce6a2dcca admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.496292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.496760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.497132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.497414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.497853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.498306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.511416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.514412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.528142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1111/2190] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:50.559255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.609912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84a778-0818-496a-ad5a-f1e9a2d9a73c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.614737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.615350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.615632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.615909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] PATH_INFO: `/v3/projects/06bae63a727d42f1a7692887066f9d6e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.617115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06bae63a727d42f1a7692887066f9d6e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.619630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.670487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:50.702737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d5b955-d3ff-46b3-872c-1c0eefa65355 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.735553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06bae63a727d42f1a7692887066f9d6e', 'request_id': 'req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.752618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06bae63a727d42f1a7692887066f9d6e', 'request_id': 'req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.761303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06bae63a727d42f1a7692887066f9d6e', 'request_id': 'req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.775445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] The token cache is being invalidate because project 06bae63a727d42f1a7692887066f9d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:50.775789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.776408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-1c5a6e3d-4c1e-4e8c-8910-5f196a666ab5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.777902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1080/2191] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:50 2022] DELETE /identity/v3/projects/06bae63a727d42f1a7692887066f9d6e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:50.782843 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a1c5c-11a5-4064-86ac-4d49cf8876c0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.783235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a1c5c-11a5-4064-86ac-4d49cf8876c0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.783746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63a1c5c-11a5-4064-86ac-4d49cf8876c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.797714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.799198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.800249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.800745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] PATH_INFO: `/v3/users/1a4d8e5164f344e384c2a525fba2caa5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.802409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4d8e5164f344e384c2a525fba2caa5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.805321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d63a1c5c-11a5-4064-86ac-4d49cf8876c0 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:50.806327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.806852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d63a1c5c-11a5-4064-86ac-4d49cf8876c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.816275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1081/2192] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:50.823897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.824281 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.824655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.846172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:50.847483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.856325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1082/2193] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:50.861060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.864557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4d8e5164f344e384c2a525fba2caa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.865403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4d8e5164f344e384c2a525fba2caa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.869993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7d51c-bb4e-4d7e-ae2a-64dc78a58423 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.873183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4d8e5164f344e384c2a525fba2caa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.873836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4d8e5164f344e384c2a525fba2caa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.874696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.875503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.875872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.876164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.876800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:21:50.877106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.878715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.886549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-98b6d1fa-f433-4cc1-b1ea-63dfec8c6cf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4d8e5164f344e384c2a525fba2caa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:50.890081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1083/2194] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:21:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:21:50.896008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.897940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.900609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1112/2195] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:21:50 2022] DELETE /identity/v3/users/1a4d8e5164f344e384c2a525fba2caa5 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:21:50.904768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.905059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.905326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.911335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ad4f2f-6817-4d93-82a8-75ac7996668d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:50.924701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:50.925886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:50.962639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:50.971583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:50.972277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:50.972566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:50.973052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:50.973903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:50.976042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:50.987346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1113/2196] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:21:50.997724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:50.999127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:51.007568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1084/2197] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:21:50 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:51.008558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:51.013044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:51.063190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c9d839-e049-4f6d-8974-ba77bb6b110c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.072862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485df455-cc4b-4c63-bd54-b80ffcd3ce5c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.077744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.079011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.079319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.079601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.080229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.082138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.104656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1085/2198] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:21:51 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:21:51.108845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:51.112450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.113168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.113546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.113920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.118662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501d6326-b826-4df5-b27e-9cf92c715940 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.123161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.123323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.123737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.124009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.124279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.124902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.125150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.126744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.134795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1114/2199] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:21:51.143424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1086/2200] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:21:51.144540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:51.154251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449817a3-ce1e-4fdb-ac0b-09899555d5e3 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.159985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce736207-dc6b-4b08-a282-f72ace7b556b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.160675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce736207-dc6b-4b08-a282-f72ace7b556b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.160992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce736207-dc6b-4b08-a282-f72ace7b556b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.161293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce736207-dc6b-4b08-a282-f72ace7b556b admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.162024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce736207-dc6b-4b08-a282-f72ace7b556b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.163844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce736207-dc6b-4b08-a282-f72ace7b556b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.178196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1115/2201] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:21:51.182733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:51.195174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7112fc5-8517-4225-8b4f-f215b1365001 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.200359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.201153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.201443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.201722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] PATH_INFO: `/v3/projects/cf765f5001bb46c8bfbd77121e5a5aa9/users/a177c0a02a824c3086dece9b435a7c27/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.219767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf765f5001bb46c8bfbd77121e5a5aa9, user_id=a177c0a02a824c3086dece9b435a7c27, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.222482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.236310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1087/2202] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:21:51 2022] PUT /identity/v3/projects/cf765f5001bb46c8bfbd77121e5a5aa9/users/a177c0a02a824c3086dece9b435a7c27/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:21:51.247724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.248104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.248447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.282266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] MFA Rules not processed for user `a177c0a02a824c3086dece9b435a7c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:21:51.283367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:51.379366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1116/2203] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:21:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:21:51.390413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:51.392369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:51.453463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf3bce8-440a-4881-a9fe-dbb9512278d0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.457850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:21:51.459264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:21:51.459755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.460423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.460712 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.461020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.471618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.473649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.486640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1088/2204] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:21:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:21:51.524265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:21:51.541555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1121e7-27fd-4f69-9661-bb97cbba31d7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:21:51.546061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:21:51.546726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:21:51.547560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:21:51.547838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] PATH_INFO: `/v3/projects/df77f2396aae4df6a5634bf18d492fca` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:21:51.548957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=df77f2396aae4df6a5634bf18d492fca)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:21:51.550924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:21:51.600159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df77f2396aae4df6a5634bf18d492fca', 'request_id': 'req-4be408e2-8d79-4319-9587-16c02a7156ab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:51.607026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df77f2396aae4df6a5634bf18d492fca', 'request_id': 'req-4be408e2-8d79-4319-9587-16c02a7156ab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:51.615383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df77f2396aae4df6a5634bf18d492fca', 'request_id': 'req-4be408e2-8d79-4319-9587-16c02a7156ab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:51.630855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] The token cache is being invalidate because project df77f2396aae4df6a5634bf18d492fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:21:51.631149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:51.631897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4be408e2-8d79-4319-9587-16c02a7156ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:21:51.633705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1117/2205] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:21:51 2022] DELETE /identity/v3/projects/df77f2396aae4df6a5634bf18d492fca => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:01.479818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:01.627026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:05.704918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:05.707098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:05.722399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225bb8ec-e816-4f28-9869-10874dfca5c9 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:05.788377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:05.789013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:05.789294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:05.789571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] PATH_INFO: `/v3/users/95fcd80347b344eb84555321908543d6` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:05.790742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=95fcd80347b344eb84555321908543d6)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:05.792667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:05.848760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95fcd80347b344eb84555321908543d6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:05.849359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95fcd80347b344eb84555321908543d6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:05.856286 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95fcd80347b344eb84555321908543d6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:05.856934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95fcd80347b344eb84555321908543d6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:05.872576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95fcd80347b344eb84555321908543d6'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:05.885908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1089/2206] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:05 2022] DELETE /identity/v3/users/95fcd80347b344eb84555321908543d6 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:05.892606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:05.893142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:05.893236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:05.921594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:05.923393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:05.994390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1118/2207] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:06.004595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:06.006138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:06.079960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef7491c-5a63-4b94-82c8-8ed10ede43dd admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:06.085759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:06.086490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:06.086838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:06.087178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:06.097878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:06.099081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:06.111172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1090/2208] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:06.455548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:06.457193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:06.469701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4bc89-383b-4a7d-8c19-118c7353bd80 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:06.474899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:06.475541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:06.475801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:06.476065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] PATH_INFO: `/v3/projects/aec251ef21974a5fa4d1fb3bcc692a5b` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:06.477182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aec251ef21974a5fa4d1fb3bcc692a5b)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:06.479210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:06.529911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aec251ef21974a5fa4d1fb3bcc692a5b', 'request_id': 'req-12c60e8e-c58f-4910-a9fb-892db591d16d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.536307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aec251ef21974a5fa4d1fb3bcc692a5b', 'request_id': 'req-12c60e8e-c58f-4910-a9fb-892db591d16d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.541967 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aec251ef21974a5fa4d1fb3bcc692a5b', 'request_id': 'req-12c60e8e-c58f-4910-a9fb-892db591d16d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.555398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] The token cache is being invalidate because project aec251ef21974a5fa4d1fb3bcc692a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:06.555688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.556323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.557837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1119/2209] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:06 2022] DELETE /identity/v3/projects/aec251ef21974a5fa4d1fb3bcc692a5b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:06.561857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:06.572006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe45118-48d0-4229-ba36-4d27f74f00b3 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:06.591583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:06.647710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:06.648483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:06.648762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:06.649040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] PATH_INFO: `/v3/users/99eb441fcc404f7a9d3e3619609d9be3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:06.650169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99eb441fcc404f7a9d3e3619609d9be3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:06.653128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:06.702417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99eb441fcc404f7a9d3e3619609d9be3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.703218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99eb441fcc404f7a9d3e3619609d9be3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.710040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99eb441fcc404f7a9d3e3619609d9be3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.710709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99eb441fcc404f7a9d3e3619609d9be3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.723494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9a47b1b9-0965-490f-887a-08b75029ea42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99eb441fcc404f7a9d3e3619609d9be3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:06.737742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1091/2210] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:06 2022] DELETE /identity/v3/users/99eb441fcc404f7a9d3e3619609d9be3 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:07.006866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.008309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.023740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c60e8e-c58f-4910-a9fb-892db591d16d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.082841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.083578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.083917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.084240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] PATH_INFO: `/v3/projects/7e2d2a9195b84d6ba7ddfe6bff8e623e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.085420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e2d2a9195b84d6ba7ddfe6bff8e623e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.087397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.120574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:07.137662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e2d2a9195b84d6ba7ddfe6bff8e623e', 'request_id': 'req-544ab018-bdbb-4add-b4c6-d1c93ac153f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.150736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e2d2a9195b84d6ba7ddfe6bff8e623e', 'request_id': 'req-544ab018-bdbb-4add-b4c6-d1c93ac153f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.157204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e2d2a9195b84d6ba7ddfe6bff8e623e', 'request_id': 'req-544ab018-bdbb-4add-b4c6-d1c93ac153f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.170676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] The token cache is being invalidate because project 7e2d2a9195b84d6ba7ddfe6bff8e623e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:07.171149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.171956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-544ab018-bdbb-4add-b4c6-d1c93ac153f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.173885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1120/2211] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:07 2022] DELETE /identity/v3/projects/7e2d2a9195b84d6ba7ddfe6bff8e623e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:07.180237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.180581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.180940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.201084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:07.202314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.269626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1092/2212] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:07.279428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.279744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.279997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.301366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:07.303828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.312824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1121/2213] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:07.319505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.330407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1eb67a-a63e-4fc1-a89c-427456ade4cf None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.335290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.336232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.336575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.337050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.337759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:07.338120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.339932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.353636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1093/2214] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:22:07.358060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.369951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566c57a-291f-4c97-80fc-2ca4159e4337 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.380432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.381023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.381443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.381718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.382409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.384301 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.418745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1122/2215] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:07.424001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.435383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d151f334-5435-47a6-9954-bec9c969225c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.463811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.466240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.469188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:07.538572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.539184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.539464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.539759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.540395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.543689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.572439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a41070-9cbb-46be-88f6-6d608fa6ef10 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.576537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1094/2216] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/users => generated 293 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:07.580622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.586505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.587602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.587976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.588316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] PATH_INFO: `/v3/users/bda3a904463c413c81ff90d51e2e9057` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.590267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bda3a904463c413c81ff90d51e2e9057)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.592573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2c0618-acbd-427f-a36b-863dc017216f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.594425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.604257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.604879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.605155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.605427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.606771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.608657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.620329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1095/2217] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:07.625572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.637525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de9da25-80b8-4fc8-9646-d7dbfbca838a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.643926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.644125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.644410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.644687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.645325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.650773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.660235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1096/2218] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:07.664894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.676278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c1e5-790c-454e-852d-2ed8d8fe1b1c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.682888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9787e152-fa19-4bbd-9b7e-a555cdc34408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.683381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e152-fa19-4bbd-9b7e-a555cdc34408 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.683776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e152-fa19-4bbd-9b7e-a555cdc34408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.684240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9787e152-fa19-4bbd-9b7e-a555cdc34408 admin admin] PATH_INFO: `/v3/projects/1d414e76296840fd965f67ff15ee67b0/users/2e2b2801b3764d53ac36b59e23c98944/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.698465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bda3a904463c413c81ff90d51e2e9057'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.700090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bda3a904463c413c81ff90d51e2e9057'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.702457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787e152-fa19-4bbd-9b7e-a555cdc34408 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d414e76296840fd965f67ff15ee67b0, user_id=2e2b2801b3764d53ac36b59e23c98944, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.705537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9787e152-fa19-4bbd-9b7e-a555cdc34408 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.713429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bda3a904463c413c81ff90d51e2e9057'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.716605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bda3a904463c413c81ff90d51e2e9057'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.722838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1097/2219] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:07 2022] PUT /identity/v3/projects/1d414e76296840fd965f67ff15ee67b0/users/2e2b2801b3764d53ac36b59e23c98944/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:07.735670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.735877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.736311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.737991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-543003fe-45b4-4c25-9970-cc763125fe7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bda3a904463c413c81ff90d51e2e9057'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:07.757026 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1123/2220] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:07 2022] DELETE /identity/v3/users/bda3a904463c413c81ff90d51e2e9057 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:07.761414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.761908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.762165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.775971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=16,keystone:SELECT=81,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:07.781515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] MFA Rules not processed for user `2e2b2801b3764d53ac36b59e23c98944`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:07.783511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.792041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:07.793780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.911474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1124/2221] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:07.922160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1098/2222] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:07.922474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.927185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.927345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.929041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:07.944213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5e223a-570d-4c1a-b93d-cf7bd20b4d05 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:07.949561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:07.950349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:07.950725 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:07.951632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:07.952381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:07.955872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:07.989932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1099/2223] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:07.993504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:07.997865 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6907f8b-2a8d-41cd-833a-e51bc47a1ec7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.045427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:08.050920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.052196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.052740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.054353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.072046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8663d6-204a-405f-ab0b-e1ff27a37253 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.076981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.077607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.077881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.078400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.079059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.080964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.103178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1100/2224] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:07 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:08.107091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.117389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a825f1-673d-4ed6-81e9-45d2c8d6328b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.122029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.122658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.122936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.123209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.123849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.125875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.125993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.127654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.136800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1101/2225] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:08.140668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1125/2226] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:08.140811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.150685 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e8dc6e-ffbd-4355-90c7-261d80470aca admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.156584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e26763e-9792-400b-8319-168f1f6ba0c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.157201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26763e-9792-400b-8319-168f1f6ba0c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.157504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26763e-9792-400b-8319-168f1f6ba0c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.157802 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e26763e-9792-400b-8319-168f1f6ba0c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.158477 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26763e-9792-400b-8319-168f1f6ba0c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.161099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e26763e-9792-400b-8319-168f1f6ba0c6 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.171129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1102/2227] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:08.175646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.186061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9362d6-d787-4198-a8e7-6df43df91049 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.192515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.193236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.193510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.193827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] PATH_INFO: `/v3/projects/0bfd46ebccc44a249469c460271eec97/users/f36146c2170b4f0d956f1008c0f8656f/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.204545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:08.214569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bfd46ebccc44a249469c460271eec97, user_id=f36146c2170b4f0d956f1008c0f8656f, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.217237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.237255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1126/2228] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:08 2022] PUT /identity/v3/projects/0bfd46ebccc44a249469c460271eec97/users/f36146c2170b4f0d956f1008c0f8656f/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:08.247562 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.247898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.248174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.282967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] MFA Rules not processed for user `f36146c2170b4f0d956f1008c0f8656f`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:08.284227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:08.367640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1103/2229] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:08.372266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.458575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5c2def-da32-4fab-9853-4b83949a2b71 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.464214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.464850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.465130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.465404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.466062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.468073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.472664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.474176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:08.488799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfa858-f9d8-4e52-a9c5-c6e4eeb4d337 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.493976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.494771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.495146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.495464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] PATH_INFO: `/v3/projects/a5184d1afd824ee3a0e9a640239c249c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.496952 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5184d1afd824ee3a0e9a640239c249c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.500176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.507380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1127/2230] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:08 2022] POST /identity/v3/projects => generated 383 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:08.511342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.595135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5184d1afd824ee3a0e9a640239c249c', 'request_id': 'req-727accdb-b5f8-42ec-ab67-f6e86123b88f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.601150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5184d1afd824ee3a0e9a640239c249c', 'request_id': 'req-727accdb-b5f8-42ec-ab67-f6e86123b88f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.605324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8e6752-8ead-40a3-9094-fc20e9faccdc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.609886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5184d1afd824ee3a0e9a640239c249c', 'request_id': 'req-727accdb-b5f8-42ec-ab67-f6e86123b88f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.621849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] The token cache is being invalidate because project a5184d1afd824ee3a0e9a640239c249c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:08.622324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.623095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.625031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1104/2231] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:08 2022] DELETE /identity/v3/projects/a5184d1afd824ee3a0e9a640239c249c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:08.633786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.633786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:08.657680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:08.686572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.688028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.688028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.688028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.688848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.690981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.714824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727accdb-b5f8-42ec-ab67-f6e86123b88f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.719782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1128/2232] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:08 2022] POST /identity/v3/users => generated 298 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:08.722273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.723342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.724052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:08.724152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.724445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.724786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] PATH_INFO: `/v3/users/417399ada57f44b49c2539ca2fc65383` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.726052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] RBAC: Authorizing `identity:delete_user(user_id=417399ada57f44b49c2539ca2fc65383)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.728159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.742446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b0f900-e5a0-4504-b9aa-13919267a037 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.747521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.747888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.748169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.748632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.749092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.751515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.754123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:08.767012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1129/2233] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:08.768946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.780312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdf365-bd5e-4882-b60d-dd0047762744 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:08.786368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:08.787911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:08.788295 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:08.788471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '417399ada57f44b49c2539ca2fc65383'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.788747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] PATH_INFO: `/v3/projects/83ed7d4705ed44a09b598fca8d7e8ba8/users/068da0fd2ef64d498554d69d533e6fc8/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:08.789153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '417399ada57f44b49c2539ca2fc65383'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.798505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '417399ada57f44b49c2539ca2fc65383'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.799431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '417399ada57f44b49c2539ca2fc65383'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.809108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ed7d4705ed44a09b598fca8d7e8ba8, user_id=068da0fd2ef64d498554d69d533e6fc8, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:08.812106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:08.816350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '417399ada57f44b49c2539ca2fc65383'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:08.832467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1105/2234] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:08 2022] DELETE /identity/v3/users/417399ada57f44b49c2539ca2fc65383 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:08.832684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1130/2235] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:08 2022] PUT /identity/v3/projects/83ed7d4705ed44a09b598fca8d7e8ba8/users/068da0fd2ef64d498554d69d533e6fc8/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:08.836689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:08.838640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:08.924834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe711e3-31dc-4f17-bc25-3851fb0babfc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.023025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2f419e-b8d0-4f10-9b54-412a2159f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.023668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f419e-b8d0-4f10-9b54-412a2159f1aa admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.023947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f419e-b8d0-4f10-9b54-412a2159f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.024223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2f419e-b8d0-4f10-9b54-412a2159f1aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.024885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f419e-b8d0-4f10-9b54-412a2159f1aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.026560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2f419e-b8d0-4f10-9b54-412a2159f1aa admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.066996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.067530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1131/2236] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:08 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:09.080084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53da77d-9ab7-40fb-9e03-8c0cbf12a72a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.085913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.086647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.086926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.087205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] PATH_INFO: `/v3/projects/83ed7d4705ed44a09b598fca8d7e8ba8/users/068da0fd2ef64d498554d69d533e6fc8/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.089159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ed7d4705ed44a09b598fca8d7e8ba8, user_id=068da0fd2ef64d498554d69d533e6fc8, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.092773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.113758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1106/2237] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:09 2022] PUT /identity/v3/projects/83ed7d4705ed44a09b598fca8d7e8ba8/users/068da0fd2ef64d498554d69d533e6fc8/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:09.125432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c44780-1841-474a-8c58-707bdb91830a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.125647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c44780-1841-474a-8c58-707bdb91830a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.126005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c44780-1841-474a-8c58-707bdb91830a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.170948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-59c44780-1841-474a-8c58-707bdb91830a None None] MFA Rules not processed for user `068da0fd2ef64d498554d69d533e6fc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:09.172726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-59c44780-1841-474a-8c58-707bdb91830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:09.215987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.218003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:09.260861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1132/2238] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:09.274168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c44780-1841-474a-8c58-707bdb91830a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.276054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-59c44780-1841-474a-8c58-707bdb91830a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:09.283447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:09.319740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef94a75-deb9-44f7-93bf-1255a9831b75 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.326443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.327392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.328346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.329193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] PATH_INFO: `/v3/projects/b60f6f591c684c26bfcb6b791d511e12` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.331346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b60f6f591c684c26bfcb6b791d511e12)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.334331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.345875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c44780-1841-474a-8c58-707bdb91830a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.350148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.350887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.351267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.351667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.353616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:09.373084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:09.394085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b60f6f591c684c26bfcb6b791d511e12', 'request_id': 'req-11e3c6fa-163b-403c-a60d-0de119093820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:09.400103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b60f6f591c684c26bfcb6b791d511e12', 'request_id': 'req-11e3c6fa-163b-403c-a60d-0de119093820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:09.410603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b60f6f591c684c26bfcb6b791d511e12', 'request_id': 'req-11e3c6fa-163b-403c-a60d-0de119093820'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:09.420621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.422261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ad12f-e332-4015-9d65-18c35219e7ca service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.423736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] The token cache is being invalidate because project b60f6f591c684c26bfcb6b791d511e12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:09.424029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:09.424699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-11e3c6fa-163b-403c-a60d-0de119093820 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:09.426492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1107/2239] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:09 2022] DELETE /identity/v3/projects/b60f6f591c684c26bfcb6b791d511e12 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:09.431205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.431681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.432391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.434238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1133/2240] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:09.455271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:09.457768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:09.530365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1108/2241] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:09.536414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.536784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.537126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.560327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:09.561642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:09.572849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1134/2242] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:09.576824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.589124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a4625b-dc58-4c74-aa17-7f80f500270b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.596141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.597445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.598234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.598711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.599579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:09.599988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.601831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.639468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1109/2243] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:22:09.644451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.656534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bce629-a325-4bca-af32-7bea0cdbcd9f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.665399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.666406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.666819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.667221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.668284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.670515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.719266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1135/2244] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:09 2022] POST /identity/v3/projects => generated 387 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:09.723404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.737503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcae1745-fe0b-47b3-98da-0730f4dfeb9c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.858162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.858162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.858162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.858162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.858580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.858580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.889875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1110/2245] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:09 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:09.890722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.902803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1865ed-1d7f-44ec-aa4e-4e7c37acc8ed admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.914769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.915480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.916230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.916575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.917480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.919411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.933007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1136/2246] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:09.937181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.947212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aee775-8606-4659-9bcd-19949c494bd9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.953751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b374434-a0ff-4a38-8662-599efd83016e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.954732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b374434-a0ff-4a38-8662-599efd83016e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.954732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b374434-a0ff-4a38-8662-599efd83016e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.954933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b374434-a0ff-4a38-8662-599efd83016e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:09.955550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b374434-a0ff-4a38-8662-599efd83016e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:09.957148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b374434-a0ff-4a38-8662-599efd83016e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:09.970008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1111/2247] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:09.975307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:09.986130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090ed46-b28d-4ca1-b7c8-5e0e2232be0e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:09.990904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:09.991494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:09.991770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:09.992041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] PATH_INFO: `/v3/projects/6c306b780068479a8a44c3dd9190d5e2/users/45f0f63fc1614b5f8f6c145f223a615e/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.034598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c306b780068479a8a44c3dd9190d5e2, user_id=45f0f63fc1614b5f8f6c145f223a615e, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.038326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.063252 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1137/2248] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:09 2022] PUT /identity/v3/projects/6c306b780068479a8a44c3dd9190d5e2/users/45f0f63fc1614b5f8f6c145f223a615e/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:10.073836 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.074097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.074418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.107302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] MFA Rules not processed for user `45f0f63fc1614b5f8f6c145f223a615e`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:10.108500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:10.118171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:10.181529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1112/2249] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:10.185612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.239642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:10.252913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc500a0-842b-4cad-a563-fe9d69390684 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.258584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.259187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.259461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.259730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.260371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.262515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.291788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1138/2250] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:10 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:10.295623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.306962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e76feb7-4c66-42be-9496-faeddef2a930 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.364365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.364365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.364647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.364863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.365488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.367404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.402133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1113/2251] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:10 2022] POST /identity/v3/users => generated 298 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:10.403405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.418680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d72be-9122-4ebf-b153-bc36addf323a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.423626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.424920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.425198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.425470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.426125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.428069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.440386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1139/2252] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:10.444591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.457790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd0008-407a-4a25-863b-d058d8d9c494 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.462814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.463511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.463785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.464058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] PATH_INFO: `/v3/projects/744ff5b4334a46afa8702b5481f64ad5/users/f30662f82ac94015aafdbfb2cfdf546a/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.484368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] RBAC: Authorizing `identity:create_grant(project_id=744ff5b4334a46afa8702b5481f64ad5, user_id=f30662f82ac94015aafdbfb2cfdf546a, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.488640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.506334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1114/2253] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:10 2022] PUT /identity/v3/projects/744ff5b4334a46afa8702b5481f64ad5/users/f30662f82ac94015aafdbfb2cfdf546a/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:10.510416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.520390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef4dc54-5b12-4211-b781-cf4f1ff88609 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.583543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a4123-e059-4457-81d2-396cc2f9c7d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.584164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a4123-e059-4457-81d2-396cc2f9c7d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.584420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a4123-e059-4457-81d2-396cc2f9c7d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.584684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a4123-e059-4457-81d2-396cc2f9c7d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.585320 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a4123-e059-4457-81d2-396cc2f9c7d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.586937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a4123-e059-4457-81d2-396cc2f9c7d0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.599808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1140/2254] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:10.603251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.621561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bba5e02-8ba7-4d5e-8b5d-1ac8cc3d1ebf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.633798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.634624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.634959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.635291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] PATH_INFO: `/v3/projects/744ff5b4334a46afa8702b5481f64ad5/users/f30662f82ac94015aafdbfb2cfdf546a/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.638326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744ff5b4334a46afa8702b5481f64ad5, user_id=f30662f82ac94015aafdbfb2cfdf546a, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.647451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.659079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1115/2255] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:10 2022] PUT /identity/v3/projects/744ff5b4334a46afa8702b5481f64ad5/users/f30662f82ac94015aafdbfb2cfdf546a/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:10.670909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.671196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.671827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.714695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] MFA Rules not processed for user `f30662f82ac94015aafdbfb2cfdf546a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:10.715946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:10.801240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1141/2256] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:10 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:10.815156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:10.817056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:10.910961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e903e-c060-4241-8d7a-5bd5df6ee692 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:10.929778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:10.931129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:10.931129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:10.931551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:10.945932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:10.948100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:10.960355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1116/2257] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:11.626266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:11.636624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:11.715681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:11.741022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc01480-3c07-4256-8553-8d569ae0339a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:11.746754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:11.747738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:11.748073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:11.748294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] PATH_INFO: `/v3/users/7c9b4978f33247aea4d5768d98120e71` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:11.749641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c9b4978f33247aea4d5768d98120e71)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:11.751991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:11.817844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c9b4978f33247aea4d5768d98120e71'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.821678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c9b4978f33247aea4d5768d98120e71'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.822713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:11.829992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c9b4978f33247aea4d5768d98120e71'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.830729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c9b4978f33247aea4d5768d98120e71'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.835174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af2fe69-85c8-421e-8679-7a20d8e99666 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:11.841283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:11.841941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:11.842266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:11.842611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] PATH_INFO: `/v3/users/45f0f63fc1614b5f8f6c145f223a615e` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:11.844483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=45f0f63fc1614b5f8f6c145f223a615e)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:11.847058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:11.849554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-033e85a9-a066-4a3f-b70b-d099bce2d98a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c9b4978f33247aea4d5768d98120e71'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.849786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:11.880105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1142/2258] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:11 2022] DELETE /identity/v3/users/7c9b4978f33247aea4d5768d98120e71 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:11.925884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45f0f63fc1614b5f8f6c145f223a615e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.926707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45f0f63fc1614b5f8f6c145f223a615e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.944116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45f0f63fc1614b5f8f6c145f223a615e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.954885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45f0f63fc1614b5f8f6c145f223a615e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.973613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45f0f63fc1614b5f8f6c145f223a615e'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:11.992349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1117/2259] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:11 2022] DELETE /identity/v3/users/45f0f63fc1614b5f8f6c145f223a615e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:12.004074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.004358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.004628 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.042467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:12.043698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.145056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.148221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.207777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1143/2260] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:12.220204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.221874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.287904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb2550-9db7-4caa-bb10-9b1a9b6440bf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.307447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.308096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.308373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.308748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] PATH_INFO: `/v3/projects/0150cb46bfdf4f50baf0ef3ecba77695` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.309291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c80801-55bd-453c-96b1-391ee7da48df None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.309821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0150cb46bfdf4f50baf0ef3ecba77695)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.313399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.315227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.316173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.316173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.316551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.326423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.328225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.338645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1144/2261] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:12.343376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.345508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.363466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:12.369032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0150cb46bfdf4f50baf0ef3ecba77695', 'request_id': 'req-d198cf23-aa43-4575-8bd2-19787e3830b8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.378650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0150cb46bfdf4f50baf0ef3ecba77695', 'request_id': 'req-d198cf23-aa43-4575-8bd2-19787e3830b8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.391860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0150cb46bfdf4f50baf0ef3ecba77695', 'request_id': 'req-d198cf23-aa43-4575-8bd2-19787e3830b8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.409698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] The token cache is being invalidate because project 0150cb46bfdf4f50baf0ef3ecba77695 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:12.410052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.411021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d198cf23-aa43-4575-8bd2-19787e3830b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.413370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1118/2262] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:12 2022] DELETE /identity/v3/projects/0150cb46bfdf4f50baf0ef3ecba77695 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:12.418957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6ef8b-b6de-48b7-9033-939bb0ff308f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.419959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6ef8b-b6de-48b7-9033-939bb0ff308f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.419959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc6ef8b-b6de-48b7-9033-939bb0ff308f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.433723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3725914-7534-4b29-a9be-cebfdc4f1ec2 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.450957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dfc6ef8b-b6de-48b7-9033-939bb0ff308f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:12.453318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dfc6ef8b-b6de-48b7-9033-939bb0ff308f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.458338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:12.509022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.509591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.509879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.510222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.512325 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.530354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1119/2263] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:12.540187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.540187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.540453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.561045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:12.562304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.571229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1120/2264] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:12.575698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.586073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a9eec2-803d-495b-88de-ab926ce0c847 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.591505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.592096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.592375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.592665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.594789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:12.594789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.596173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.598551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.599998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.609100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1145/2265] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:12.609823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1121/2266] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:22:12.614095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.633627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff919c0-71a9-45ff-b924-7b521e074d79 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.639107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.639729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.640916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.641043 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.641790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.643765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.672386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.674309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.679016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1146/2267] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:12 2022] POST /identity/v3/projects => generated 373 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:12.683004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.697349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306903b8-5719-4e3e-a7cd-cb36a6b1d981 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.776801 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de05bd-e3a8-4e25-b314-48f1f1561c79 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.782441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.783062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.783345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.783632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.784224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.786153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.788603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.789378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.789607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.789885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] PATH_INFO: `/v3/projects/6c306b780068479a8a44c3dd9190d5e2` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.791165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c306b780068479a8a44c3dd9190d5e2)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.793061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.796049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:12.821196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1147/2268] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:12 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:12.825812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.841975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7a45a-05a0-4116-b6dc-9a2fe653ba3c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.850277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.850898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.851178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.851451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.852106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.853719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.866824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1148/2269] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:12.871484 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.874237 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c306b780068479a8a44c3dd9190d5e2', 'request_id': 'req-58dff399-fe31-4251-be38-e5bee4023d78'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.880909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c306b780068479a8a44c3dd9190d5e2', 'request_id': 'req-58dff399-fe31-4251-be38-e5bee4023d78'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.885963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348839d6-5cba-4f72-9e07-9192949775ce admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:12.896742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c306b780068479a8a44c3dd9190d5e2', 'request_id': 'req-58dff399-fe31-4251-be38-e5bee4023d78'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.927128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] The token cache is being invalidate because project 6c306b780068479a8a44c3dd9190d5e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:12.927470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.928306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:12.930080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1122/2270] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:12 2022] DELETE /identity/v3/projects/6c306b780068479a8a44c3dd9190d5e2 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:12.933611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.935039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:12.957339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:12.976256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:12.977580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:12.977915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:12.978614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:12.979465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:12.981512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:12.994246 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1149/2271] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:12 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:12.996406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:12.998103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.008323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dff399-fe31-4251-be38-e5bee4023d78 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:13.011034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65e88b6-ab0c-47fe-b562-d504ce58d5a2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:13.013014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:13.013713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.013945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.014262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] PATH_INFO: `/v3/users/f30662f82ac94015aafdbfb2cfdf546a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.016120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f30662f82ac94015aafdbfb2cfdf546a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:13.016386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fb260d-a78d-4beb-8923-374b220079ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:13.016992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb260d-a78d-4beb-8923-374b220079ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.017273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb260d-a78d-4beb-8923-374b220079ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.017570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fb260d-a78d-4beb-8923-374b220079ea admin admin] PATH_INFO: `/v3/projects/1c3db89ab752467a9eb9792be3c9d2fc/users/0e3ae04716404ea3b2ff1931841e765f/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.018232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:13.040723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb260d-a78d-4beb-8923-374b220079ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c3db89ab752467a9eb9792be3c9d2fc, user_id=0e3ae04716404ea3b2ff1931841e765f, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:13.043424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fb260d-a78d-4beb-8923-374b220079ea admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:13.061472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1150/2272] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:12 2022] PUT /identity/v3/projects/1c3db89ab752467a9eb9792be3c9d2fc/users/0e3ae04716404ea3b2ff1931841e765f/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:13.073978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f30662f82ac94015aafdbfb2cfdf546a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.074654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f30662f82ac94015aafdbfb2cfdf546a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.074902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.075333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.075603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.083186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f30662f82ac94015aafdbfb2cfdf546a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.084190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f30662f82ac94015aafdbfb2cfdf546a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.102509 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f30662f82ac94015aafdbfb2cfdf546a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.106863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] MFA Rules not processed for user `0e3ae04716404ea3b2ff1931841e765f`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:13.108155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.122242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1123/2273] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:12 2022] DELETE /identity/v3/users/f30662f82ac94015aafdbfb2cfdf546a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:13.196485 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1151/2274] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:13.210619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:13.212073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.319294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eba006a-5e82-4483-8ef9-d9c8968cef9e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:13.346162 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:13.346821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.347105 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.347380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.396202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:13.397635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.407927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:13.447291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:13.450127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:13.460892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:13.461092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1124/2275] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:13.495656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f549d38b-cc28-415d-83e0-af48bde07d41 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:13.505738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:13.507175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.579377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:13.580323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.580637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.580904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.582758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.611365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e4a4ff-b4db-434f-b791-650a0483d255 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:13.632595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:13.633609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.634019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.634876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] PATH_INFO: `/v3/projects/744ff5b4334a46afa8702b5481f64ad5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.635848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=744ff5b4334a46afa8702b5481f64ad5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:13.638240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:13.664778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:13.667339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:13.678769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1152/2276] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:13.711252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744ff5b4334a46afa8702b5481f64ad5', 'request_id': 'req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.714901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:13.716374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.719223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744ff5b4334a46afa8702b5481f64ad5', 'request_id': 'req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.732963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744ff5b4334a46afa8702b5481f64ad5', 'request_id': 'req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.734017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5b7934-697a-4f4d-b3d1-54f5a4d6222f service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:13.746844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] The token cache is being invalidate because project 744ff5b4334a46afa8702b5481f64ad5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:13.747296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.749102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-438c1e5c-6cc0-4beb-9db2-a32161d2d5ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.751431 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1125/2277] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:13 2022] DELETE /identity/v3/projects/744ff5b4334a46afa8702b5481f64ad5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:13.756593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1897a-1e1b-4a91-8d08-beaa3f909fd1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.756799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1897a-1e1b-4a91-8d08-beaa3f909fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.757071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d1897a-1e1b-4a91-8d08-beaa3f909fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.807855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:13.808512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.808792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.809068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] PATH_INFO: `/v3/users/0e3ae04716404ea3b2ff1931841e765f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.810281 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e3ae04716404ea3b2ff1931841e765f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:13.813356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:13.870906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e3ae04716404ea3b2ff1931841e765f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.871780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e3ae04716404ea3b2ff1931841e765f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.873112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:13.881123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e3ae04716404ea3b2ff1931841e765f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.882126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e3ae04716404ea3b2ff1931841e765f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.896843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-03ff0b4b-17eb-4a6b-a6b7-e67068a96faa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e3ae04716404ea3b2ff1931841e765f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:13.912134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1153/2278] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:13 2022] DELETE /identity/v3/users/0e3ae04716404ea3b2ff1931841e765f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:13.916425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.916846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.917126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:13.917456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a6d1897a-1e1b-4a91-8d08-beaa3f909fd1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:13.918689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a6d1897a-1e1b-4a91-8d08-beaa3f909fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.938805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:13.939879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:13.983414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1126/2279] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:13.992410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:13.992922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:13.993248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.010415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:14.017601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:14.018585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1154/2280] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:14.020601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.031182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.033143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.033914 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1127/2281] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:14.038221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.049702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17365789-97c2-4a20-aa99-8534c0e0083d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.058493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.059166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.059370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.059656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.060679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:14.061012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.063743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.081558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1128/2282] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:22:14.082511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.095483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5fba68-1d37-4a7e-9470-f3a0b0bd560c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.101652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.102337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.102670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.102950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.103056 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762a65c6-d682-4d22-a7f2-4c30c9c5daab None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.103724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.107595 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.108206 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.108477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.108930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.110581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.143834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.145384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.147941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1129/2283] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/projects => generated 385 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:14.152433 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.160244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1155/2284] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:14.217983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b206a3ef-d94e-48cd-9511-6020c97b5256 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.223338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.224012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.224393 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.224567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.225437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.227986 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.251316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1130/2285] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:14.255654 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.266570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a233dda3-46af-4157-b576-602e0ea8e4e5 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.273052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.273729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.274123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.274458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.275242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.277162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.288559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1156/2286] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:14.292591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.302689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6c401a-d40c-4283-b0f9-8d105aa61bea admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.307277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87a6107-98c2-4532-a363-8565e5fa2483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.307871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a6107-98c2-4532-a363-8565e5fa2483 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.308147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a6107-98c2-4532-a363-8565e5fa2483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.308417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87a6107-98c2-4532-a363-8565e5fa2483 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.309045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a6107-98c2-4532-a363-8565e5fa2483 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.310658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87a6107-98c2-4532-a363-8565e5fa2483 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.321464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1131/2287] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:14.326756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.338065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3296fa-8c13-48b6-b859-481c480a6582 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.343514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.344562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.344706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.344977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] PATH_INFO: `/v3/projects/7aa2df867ca54a0e9ae758e176c914b0/users/c13944d5f59245999657a257240dfa9b/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.360011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa2df867ca54a0e9ae758e176c914b0, user_id=c13944d5f59245999657a257240dfa9b, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.363072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.376176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1157/2288] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:14 2022] PUT /identity/v3/projects/7aa2df867ca54a0e9ae758e176c914b0/users/c13944d5f59245999657a257240dfa9b/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:14.387749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.388054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.388324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.416969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] MFA Rules not processed for user `c13944d5f59245999657a257240dfa9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:14.418174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.469437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.471635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.504686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1132/2289] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:14.524387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.525885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.543867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:14.573324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:14.586378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fa3b8-1848-4da0-8fe4-134a9fcc5475 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.591604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.592231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.592504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.592774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] PATH_INFO: `/v3/projects/1c3db89ab752467a9eb9792be3c9d2fc` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.594857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c3db89ab752467a9eb9792be3c9d2fc)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.596788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.614136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b34ddc-0352-43b0-a3c8-46d7e4d76cc6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.624038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be267b0c-a83d-4ce4-8616-ba579152f2d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.624735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267b0c-a83d-4ce4-8616-ba579152f2d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.625159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267b0c-a83d-4ce4-8616-ba579152f2d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.625517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be267b0c-a83d-4ce4-8616-ba579152f2d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.636667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267b0c-a83d-4ce4-8616-ba579152f2d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.638780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be267b0c-a83d-4ce4-8616-ba579152f2d2 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.651031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1133/2290] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:14.659439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c3db89ab752467a9eb9792be3c9d2fc', 'request_id': 'req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:14.667143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c3db89ab752467a9eb9792be3c9d2fc', 'request_id': 'req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:14.673063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c3db89ab752467a9eb9792be3c9d2fc', 'request_id': 'req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:14.689848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] The token cache is being invalidate because project 1c3db89ab752467a9eb9792be3c9d2fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:14.690377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:14.692420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-56e39a05-bb89-4c69-887e-9d3d7c91e8e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:14.694392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1158/2291] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:14 2022] DELETE /identity/v3/projects/1c3db89ab752467a9eb9792be3c9d2fc => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:14.699176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.699553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.699920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.723268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:14.724530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.787068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1134/2292] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:14.794898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.795258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.795630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.817900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:14.819231 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:14.828782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1159/2293] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:14.833144 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.846887 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc360d07-f8ac-42e2-b645-675f85e5d6a6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.856651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.856651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.856651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.856651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.857025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:14.857025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.857025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.871116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1135/2294] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:22:14.876613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.888990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206c1c56-f814-448f-a9ae-4609eb8b2d4b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:14.894920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:14.895640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:14.895859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:14.896130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:14.896767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:14.898781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:14.946876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1160/2295] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/projects => generated 363 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:14.951243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:14.962821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd92b36b-5737-43e1-9fae-db2798469721 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.046910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.047580 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.047967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.048289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.049037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.051501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.073824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1136/2296] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:14 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:15.077942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.089503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db04cab-57d1-494e-a51d-d22e860a2ad0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.095358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.095984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.096258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.096527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.097166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.098939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.109996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1161/2297] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:15.113985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.125565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2cf6c7-371c-4a5a-a5e6-80d6131429e2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.129886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9173a05c-5b1a-48b6-a6c5-b4e70e321fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.130508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173a05c-5b1a-48b6-a6c5-b4e70e321fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.130785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173a05c-5b1a-48b6-a6c5-b4e70e321fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.131171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9173a05c-5b1a-48b6-a6c5-b4e70e321fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.131811 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173a05c-5b1a-48b6-a6c5-b4e70e321fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.134492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9173a05c-5b1a-48b6-a6c5-b4e70e321fa5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.147246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1137/2298] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:15.151299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.179165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84508add-1625-42c1-81ae-eb0cd39edf91 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.185508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.187392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.187392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.187655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] PATH_INFO: `/v3/projects/8e94b7a633db45589b640638adc1a01d/users/bf038093bc2c498ca3a206e18eac8ad6/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.211742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e94b7a633db45589b640638adc1a01d, user_id=bf038093bc2c498ca3a206e18eac8ad6, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.214930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.246506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1162/2299] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:15 2022] PUT /identity/v3/projects/8e94b7a633db45589b640638adc1a01d/users/bf038093bc2c498ca3a206e18eac8ad6/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:15.254871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.255315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.255642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.292598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] MFA Rules not processed for user `bf038093bc2c498ca3a206e18eac8ad6`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:15.293930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:15.325858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:15.362944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1138/2300] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:15.367491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.430037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaff721-08f6-4c59-9f4d-8705321b5f47 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.434787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.435380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.435653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.435921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.436542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.439132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.443849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:15.467125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1163/2301] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:15 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:15.471549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.481784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d747ffc-cab6-4129-9163-9036f1d50135 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.535654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.536241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.536515 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.536794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.537404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.539318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.560417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1139/2302] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:15 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:15.565241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.575397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9a44e-5f30-4762-bde1-4d9ba4b09ffe admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.580245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.580835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.581112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.581383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.582018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.583651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.593601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1164/2303] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:15.597839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.609657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40834075-f23d-45f6-8e93-5b5c5d0dfb07 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.614146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.614827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.615100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.615371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] PATH_INFO: `/v3/projects/f00370aa18b546e0b0f765918ac9345a/users/b2e169b3545f4edb8847cf904cb8a104/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.631140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00370aa18b546e0b0f765918ac9345a, user_id=b2e169b3545f4edb8847cf904cb8a104, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.633748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.646924 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1140/2304] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:15 2022] PUT /identity/v3/projects/f00370aa18b546e0b0f765918ac9345a/users/b2e169b3545f4edb8847cf904cb8a104/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:15.650853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.660712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fb6e7-0013-4987-afd5-35fb6aaeaa3f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.716388 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4eb02-2b09-4b71-9b87-febe5d489465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.717119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4eb02-2b09-4b71-9b87-febe5d489465 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.717229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4eb02-2b09-4b71-9b87-febe5d489465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.717436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4eb02-2b09-4b71-9b87-febe5d489465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.718633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4eb02-2b09-4b71-9b87-febe5d489465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.720647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b4eb02-2b09-4b71-9b87-febe5d489465 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.731147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1165/2305] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:15 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:15.736809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.751998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7114339f-c808-47b9-beee-0c77be1c788f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.757992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.758689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.758972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.759249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] PATH_INFO: `/v3/projects/f00370aa18b546e0b0f765918ac9345a/users/b2e169b3545f4edb8847cf904cb8a104/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.761199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f00370aa18b546e0b0f765918ac9345a, user_id=b2e169b3545f4edb8847cf904cb8a104, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.763921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.775560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1141/2306] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:15 2022] PUT /identity/v3/projects/f00370aa18b546e0b0f765918ac9345a/users/b2e169b3545f4edb8847cf904cb8a104/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:15.788979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.789361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.789663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.819208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] MFA Rules not processed for user `b2e169b3545f4edb8847cf904cb8a104`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:15.820408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:15.861472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.863312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:15.891675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1166/2307] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:15 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:15.902957 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:15.904818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:15.930223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e882cecb-7af7-4f0d-a1f7-ce06fc958561 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.935110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.935885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.936156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.936421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] PATH_INFO: `/v3/users/a177c0a02a824c3086dece9b435a7c27` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.937627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a177c0a02a824c3086dece9b435a7c27)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.939543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.962662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d05ca7-5654-4d58-b23c-3fda6f31c847 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:15.966623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:15.967514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e363d7-c2d6-4719-8ff9-6d6763e900f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:15.969274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e363d7-c2d6-4719-8ff9-6d6763e900f9 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:15.969699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e363d7-c2d6-4719-8ff9-6d6763e900f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:15.970020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e363d7-c2d6-4719-8ff9-6d6763e900f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:15.980875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e363d7-c2d6-4719-8ff9-6d6763e900f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:15.982342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e363d7-c2d6-4719-8ff9-6d6763e900f9 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:15.986980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:15.989696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a177c0a02a824c3086dece9b435a7c27'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:15.990409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a177c0a02a824c3086dece9b435a7c27'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:15.994972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1167/2308] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:15.999138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a177c0a02a824c3086dece9b435a7c27'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:15.999839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a177c0a02a824c3086dece9b435a7c27'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.015722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a177c0a02a824c3086dece9b435a7c27'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.035830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1142/2309] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:15 2022] DELETE /identity/v3/users/a177c0a02a824c3086dece9b435a7c27 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:16.040701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.041186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.041504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.063328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:16.066741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.138126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1168/2310] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:16.151559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.153099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.225664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42fb3e-8811-445c-8e4a-62c8ddc9a07e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:16.239845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:16.239845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.239845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.239845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.246250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:16.247941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:16.259679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1143/2311] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:16.550428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.552038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.567045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae992ade-3b8d-4347-8bcb-ddaf015f393f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:16.572538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:16.573145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.573446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.573730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] PATH_INFO: `/v3/users/bf038093bc2c498ca3a206e18eac8ad6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.575475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf038093bc2c498ca3a206e18eac8ad6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:16.578876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:16.611436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.613118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.630822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2e8568-587d-4116-86ca-8f10814ba3f1 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:16.637051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:16.637733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.638009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.638320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] PATH_INFO: `/v3/projects/cf765f5001bb46c8bfbd77121e5a5aa9` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.639447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf765f5001bb46c8bfbd77121e5a5aa9)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:16.642073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:16.692287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf038093bc2c498ca3a206e18eac8ad6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.693312 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf038093bc2c498ca3a206e18eac8ad6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.701596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf765f5001bb46c8bfbd77121e5a5aa9', 'request_id': 'req-6b2b054f-2e03-40ac-a626-05aaf368c6e8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.703536 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf038093bc2c498ca3a206e18eac8ad6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.704308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf038093bc2c498ca3a206e18eac8ad6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.708415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf765f5001bb46c8bfbd77121e5a5aa9', 'request_id': 'req-6b2b054f-2e03-40ac-a626-05aaf368c6e8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.719341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf765f5001bb46c8bfbd77121e5a5aa9', 'request_id': 'req-6b2b054f-2e03-40ac-a626-05aaf368c6e8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.720343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d88409ea-1a37-4ae0-b4ae-ee638bc7da84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf038093bc2c498ca3a206e18eac8ad6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.735094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1169/2312] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:16 2022] DELETE /identity/v3/users/bf038093bc2c498ca3a206e18eac8ad6 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:16.736159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] The token cache is being invalidate because project cf765f5001bb46c8bfbd77121e5a5aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:16.736608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.738418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6b2b054f-2e03-40ac-a626-05aaf368c6e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:16.739597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.739891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.740196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.741190 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1144/2313] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:16 2022] DELETE /identity/v3/projects/cf765f5001bb46c8bfbd77121e5a5aa9 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:16.747837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf4f9e-5cce-4c62-a15d-bdeeb7edd274 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.748126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf4f9e-5cce-4c62-a15d-bdeeb7edd274 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.748426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf4f9e-5cce-4c62-a15d-bdeeb7edd274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.763416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:16.764707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.769889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-aecf4f9e-5cce-4c62-a15d-bdeeb7edd274 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:16.771078 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-aecf4f9e-5cce-4c62-a15d-bdeeb7edd274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.800867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:16.832919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1170/2314] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:16.839918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1145/2315] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:16.842328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.844455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.848351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.848729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.849007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.857620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:16.872623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:16.873913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:16.882038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1146/2316] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:16.886094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.896072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38c7b40-18bc-4411-aba0-6093c9e271b8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:16.900842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:16.901346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.901620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.902042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.902855 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:16.903151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:16.904753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:16.905138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee7d391-1ee4-467c-a5be-f040a919cf31 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:16.909936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:16.910579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.910856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.911126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.917069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1147/2317] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:22:16.922112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.923334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:16.924819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:16.932173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286228e2-a41e-4cee-ab3b-749a4c8daf95 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:16.934781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1171/2318] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:16.937626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:16.938260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:16.938523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:16.938788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:16.939406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:16.941291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:16.968529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1148/2319] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:16 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:16.973556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:16.983245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034bcfa-0544-4c28-acef-9f031f8e2882 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.046219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.047441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.047842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.048119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.048876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.051004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.076848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1172/2320] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:16 2022] POST /identity/v3/users => generated 289 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:17.079258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.090740 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259a39d1-948e-4028-a1c8-cb1b54bdc075 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.096323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.096936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.097235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.097821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.098101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.099716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.110044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1149/2321] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:17.114241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.124794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490fb118-ec08-4978-8629-49874251ac05 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.129410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f73d2f-cde7-44ae-9684-2c57d8d15f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.130007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73d2f-cde7-44ae-9684-2c57d8d15f6d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.130319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73d2f-cde7-44ae-9684-2c57d8d15f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.130594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f73d2f-cde7-44ae-9684-2c57d8d15f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.131296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f73d2f-cde7-44ae-9684-2c57d8d15f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.132883 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f73d2f-cde7-44ae-9684-2c57d8d15f6d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.143894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1173/2322] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:17.148080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.158746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ed37bc-a3a2-4c14-aa0d-d5b70ff75972 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.163355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.164002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.164272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.164546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] PATH_INFO: `/v3/projects/b80fc33076004182bcac3cd05dd7a262/users/499c907a7c5041db8456b97608869a96/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.179333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b80fc33076004182bcac3cd05dd7a262, user_id=499c907a7c5041db8456b97608869a96, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.182143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.197158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1150/2323] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:17 2022] PUT /identity/v3/projects/b80fc33076004182bcac3cd05dd7a262/users/499c907a7c5041db8456b97608869a96/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:17.207868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.208185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.208522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.220308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.222479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.245362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] MFA Rules not processed for user `499c907a7c5041db8456b97608869a96`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:17.248306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.316247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f0cdc-ff4b-4cab-a744-d95ca5f9b7f5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.321583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.322227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.322541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.322813 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] PATH_INFO: `/v3/projects/8e94b7a633db45589b640638adc1a01d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.323917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e94b7a633db45589b640638adc1a01d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.324225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1174/2324] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:17.325793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.328627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.339013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea1c09a-5502-4342-89a3-57500df85e9b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.344511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.345149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.345427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.345697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.346355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.348266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.370734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:17.376128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e94b7a633db45589b640638adc1a01d', 'request_id': 'req-40839e75-f1e4-40b0-86c7-b92c4597939b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.381588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1175/2325] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:17 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:17.381747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:17.382484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e94b7a633db45589b640638adc1a01d', 'request_id': 'req-40839e75-f1e4-40b0-86c7-b92c4597939b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.383076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.390841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e94b7a633db45589b640638adc1a01d', 'request_id': 'req-40839e75-f1e4-40b0-86c7-b92c4597939b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.403875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] The token cache is being invalidate because project 8e94b7a633db45589b640638adc1a01d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:17.404160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.405194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.407339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1151/2326] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:17 2022] DELETE /identity/v3/projects/8e94b7a633db45589b640638adc1a01d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:17.411201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.412533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.447230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a9c32-8132-482f-859a-db0e491e528a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.451770 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.452399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.452666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.452932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.453549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.455508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.477668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1176/2327] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:17 2022] POST /identity/v3/users => generated 288 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:17.479402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40839e75-f1e4-40b0-86c7-b92c4597939b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.481415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.483060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.484125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.484891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.485169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.485441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] PATH_INFO: `/v3/users/b2e169b3545f4edb8847cf904cb8a104` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.486583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2e169b3545f4edb8847cf904cb8a104)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.488480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.495342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6ce95-b356-4871-ac51-91edefaeef18 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.500682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.501471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.501749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.502017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.502701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.504323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.514551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1177/2328] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:17.519201 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.529257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ad5cab-2376-4aa0-8a59-38d8b3b310fd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.534888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.535500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.535781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.536107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.536777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.538452 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.538583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2e169b3545f4edb8847cf904cb8a104'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.539189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2e169b3545f4edb8847cf904cb8a104'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.547094 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2e169b3545f4edb8847cf904cb8a104'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.547974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2e169b3545f4edb8847cf904cb8a104'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.549925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1178/2329] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:17.555376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.561249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.567536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b0ef142f-76d7-42f2-8966-34ce8e1ef6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2e169b3545f4edb8847cf904cb8a104'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:17.574742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7ed199-4fa7-4b51-959a-80b72fc02d8a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.579486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.580137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.580414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.580683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] PATH_INFO: `/v3/projects/4774e509fb8b4e33b356b5246e8c6a9a/users/e0e693d4933341edb5a8b322ce3eaa82/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.581308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1152/2330] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:17 2022] DELETE /identity/v3/users/b2e169b3545f4edb8847cf904cb8a104 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:17.596127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] RBAC: Authorizing `identity:create_grant(project_id=4774e509fb8b4e33b356b5246e8c6a9a, user_id=e0e693d4933341edb5a8b322ce3eaa82, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.598812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.613976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1179/2331] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:17 2022] PUT /identity/v3/projects/4774e509fb8b4e33b356b5246e8c6a9a/users/e0e693d4933341edb5a8b322ce3eaa82/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:17.625969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.626293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.626961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.659506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] MFA Rules not processed for user `e0e693d4933341edb5a8b322ce3eaa82`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:17.661026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.713275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:17.742857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1153/2332] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:17.744852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.834275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c27d0b7-791a-43e4-b5ff-5f916e737fde admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.834275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.834275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.834275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.834971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.834971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.836776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.868840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1180/2333] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:17 2022] POST /identity/v3/projects => generated 375 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:17.872686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.883181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b1ce2-9b44-4d84-a8ad-866cb8bb4ec8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.919420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.921909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:17.945426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.945951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.946250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.946527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.947170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.949111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:17.966968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:17.973827 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1154/2334] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:17 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:17.977534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:17.988410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd9ed0-c625-4dce-b5da-93cab8b7fa57 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:17.994378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:17.994968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:17.995247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:17.995527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:17.996158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:17.997746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.004524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc6aca9-434a-4801-90d3-a6c5ce7c6ccb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.011615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1155/2335] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:18.011775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.011775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.011775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.011775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] PATH_INFO: `/v3/projects/f00370aa18b546e0b0f765918ac9345a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.012090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f00370aa18b546e0b0f765918ac9345a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.013209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.013570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.024903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbb1b67-2313-47cd-b26d-65ce8de666b3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.029498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.030219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.030565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.030841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] PATH_INFO: `/v3/projects/71a387f8a0534afe9d9c8205f0a07ac3/users/20444801fc614006be1d69e7c62015f0/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.046115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a387f8a0534afe9d9c8205f0a07ac3, user_id=20444801fc614006be1d69e7c62015f0, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.049208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.064274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1156/2336] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:18 2022] PUT /identity/v3/projects/71a387f8a0534afe9d9c8205f0a07ac3/users/20444801fc614006be1d69e7c62015f0/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:18.066799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f00370aa18b546e0b0f765918ac9345a', 'request_id': 'req-4f3b2463-0704-4c79-b5bb-da2937a62916'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:18.068128 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.073207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f00370aa18b546e0b0f765918ac9345a', 'request_id': 'req-4f3b2463-0704-4c79-b5bb-da2937a62916'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:18.079763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f00370aa18b546e0b0f765918ac9345a', 'request_id': 'req-4f3b2463-0704-4c79-b5bb-da2937a62916'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:18.093315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] The token cache is being invalidate because project f00370aa18b546e0b0f765918ac9345a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:18.093606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:18.094544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f3b2463-0704-4c79-b5bb-da2937a62916 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:18.096083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1181/2337] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:17 2022] DELETE /identity/v3/projects/f00370aa18b546e0b0f765918ac9345a => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:18.101232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc936c-1e73-485f-98dc-5ad3a4859453 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.101527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc936c-1e73-485f-98dc-5ad3a4859453 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.101937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdc936c-1e73-485f-98dc-5ad3a4859453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.123385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-afdc936c-1e73-485f-98dc-5ad3a4859453 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:18.124639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-afdc936c-1e73-485f-98dc-5ad3a4859453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.136848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093db1f3-b4b1-44fb-864f-953e4ad1488d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.142112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.142912 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.143182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.143445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.144080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.145643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.155409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1157/2338] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:18.159640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.161314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.173865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b10224-3016-4282-a0d3-4e51aba12b3a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.178542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff5d50-a24f-4774-b6f4-65053d4e7b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.179179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff5d50-a24f-4774-b6f4-65053d4e7b21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.179456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff5d50-a24f-4774-b6f4-65053d4e7b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.179730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff5d50-a24f-4774-b6f4-65053d4e7b21 admin admin] PATH_INFO: `/v3/projects/71a387f8a0534afe9d9c8205f0a07ac3/users/20444801fc614006be1d69e7c62015f0/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.181586 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff5d50-a24f-4774-b6f4-65053d4e7b21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a387f8a0534afe9d9c8205f0a07ac3, user_id=20444801fc614006be1d69e7c62015f0, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.184274 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff5d50-a24f-4774-b6f4-65053d4e7b21 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.190879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1182/2339] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:18.195109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1158/2340] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:18 2022] PUT /identity/v3/projects/71a387f8a0534afe9d9c8205f0a07ac3/users/20444801fc614006be1d69e7c62015f0/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:18.198872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0b73-2730-48aa-8855-78389810263d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.199155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0b73-2730-48aa-8855-78389810263d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.199423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b0b73-2730-48aa-8855-78389810263d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.205799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272008-9fd4-448c-86f3-316714323f6b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.206084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272008-9fd4-448c-86f3-316714323f6b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.206386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12272008-9fd4-448c-86f3-316714323f6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.219963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-634b0b73-2730-48aa-8855-78389810263d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:18.221153 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-634b0b73-2730-48aa-8855-78389810263d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.235600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-12272008-9fd4-448c-86f3-316714323f6b None None] MFA Rules not processed for user `20444801fc614006be1d69e7c62015f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:18.236777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-12272008-9fd4-448c-86f3-316714323f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.253503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:18.276159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:18.286929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1183/2341] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:18.293304 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b0b73-2730-48aa-8855-78389810263d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.302950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b0b73-2730-48aa-8855-78389810263d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.308596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.309199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.309461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.309926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.310411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:18.310713 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.312383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.320244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1159/2342] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:18.324138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1184/2343] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:22:18.328620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272008-9fd4-448c-86f3-316714323f6b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.329983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-12272008-9fd4-448c-86f3-316714323f6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.333698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.345229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e914c8d4-53a0-42ec-b51b-2bb418f71b84 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.349821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.350446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.350725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.351253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.351877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.354136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.388291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1185/2344] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:18.390082 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.392794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12272008-9fd4-448c-86f3-316714323f6b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.441230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.441832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.442106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.442404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.444175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.453684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a0b8cd-062f-43d5-8037-76536ea0ef9c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.458355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.458953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.459238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.459535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.460162 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.462090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.488160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1186/2345] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:18.489177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.498544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa159c-cf36-4834-adad-52ac808fc895 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.503975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.504138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.504584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.504880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.505145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.505441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.505772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.507496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.514663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1160/2346] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:18.517870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1187/2347] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:18.521590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.531931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6a395-6c79-4726-b15d-1e22c68b9b72 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.536384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc88480-aa3e-4c3e-aa54-43ac4a577570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.536968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc88480-aa3e-4c3e-aa54-43ac4a577570 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.537246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc88480-aa3e-4c3e-aa54-43ac4a577570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.537519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc88480-aa3e-4c3e-aa54-43ac4a577570 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.538175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc88480-aa3e-4c3e-aa54-43ac4a577570 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.539759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc88480-aa3e-4c3e-aa54-43ac4a577570 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.550061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1161/2348] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:18.554359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.564870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c66a8-dfcb-4068-ae47-cf2c7087d143 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.569418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.570060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.570403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.570682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] PATH_INFO: `/v3/projects/a2f367ccfb42458da4248644706b25e5/users/3a2ebc10e70843d3b25d997e95d424cb/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.585467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2f367ccfb42458da4248644706b25e5, user_id=3a2ebc10e70843d3b25d997e95d424cb, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.588140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.602135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1188/2349] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:18 2022] PUT /identity/v3/projects/a2f367ccfb42458da4248644706b25e5/users/3a2ebc10e70843d3b25d997e95d424cb/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:18.613659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.613996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.614324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.641317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:18.644775 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] MFA Rules not processed for user `3a2ebc10e70843d3b25d997e95d424cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:18.646718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:18.712479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1162/2350] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:18.718414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.788439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649e329-abfd-4dee-9dbe-d5adbe8941df admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.793190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.793790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.794067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.794479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.795118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.797094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.836372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1189/2351] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:18.837833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.850038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a62ae7-9827-4ede-aaba-ac32d7b32885 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.936392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.937607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.937886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.938151 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.938818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.941492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:18.972779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1163/2352] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:18 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:18.974424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:18.981183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:18.985346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182144ba-b240-486e-ac30-d1ddf47b3ebf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:18.991422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:18.992051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:18.992326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:18.992594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:18.993255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:18.994977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.007006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1190/2353] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:19.011111 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.022288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fdbbfa-264a-4241-ad13-2cfa0e7d7e00 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.027130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d3eca2-115d-4ef6-b41a-971ea0478054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.027776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3eca2-115d-4ef6-b41a-971ea0478054 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.028052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3eca2-115d-4ef6-b41a-971ea0478054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.028320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d3eca2-115d-4ef6-b41a-971ea0478054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.028945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3eca2-115d-4ef6-b41a-971ea0478054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.030834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d3eca2-115d-4ef6-b41a-971ea0478054 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.040925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1164/2354] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:19.044991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.055738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eb7584-3da8-44b4-8f7c-b95b3c0b239e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.060355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.060994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.061272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.061563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] PATH_INFO: `/v3/projects/817c79d7a4c749a18ab265a45257ee19/users/c9e53f3bd3124718aee1d42c203301f8/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.076270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=817c79d7a4c749a18ab265a45257ee19, user_id=c9e53f3bd3124718aee1d42c203301f8, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.079072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.091425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1191/2355] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:19 2022] PUT /identity/v3/projects/817c79d7a4c749a18ab265a45257ee19/users/c9e53f3bd3124718aee1d42c203301f8/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:19.103069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25202018-c547-474f-8043-527fdaf11724 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.103352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25202018-c547-474f-8043-527fdaf11724 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.103625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25202018-c547-474f-8043-527fdaf11724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.131556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-25202018-c547-474f-8043-527fdaf11724 None None] MFA Rules not processed for user `c9e53f3bd3124718aee1d42c203301f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:19.132743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-25202018-c547-474f-8043-527fdaf11724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:19.194369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1165/2356] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:19.198875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.264939 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4140bc4a-8db9-4a36-b0e8-badc33c628b5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.271199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.271961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.272314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.272690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.273525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.275888 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.310510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1192/2357] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:19 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:19.313069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25202018-c547-474f-8043-527fdaf11724 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.323370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25202018-c547-474f-8043-527fdaf11724 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.331452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:19.381796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.382436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.382715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.382985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.383615 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.385494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.406250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1166/2358] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:19 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:19.410136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.420198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac9f8c-4c52-47a7-b23f-b35b48ba0d33 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.425765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.426402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.426679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.426951 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.427587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.429182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.440284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1193/2359] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:19.444368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.454470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06367b41-6aaf-4b6c-a4a2-78d48461349c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.459000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.459636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.459907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.460177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] PATH_INFO: `/v3/projects/6cb4220fc54b4afea0ef5bd16491a38b/users/eab92b5d6d2e4839a26836cd915b3aa7/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.475178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb4220fc54b4afea0ef5bd16491a38b, user_id=eab92b5d6d2e4839a26836cd915b3aa7, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.477776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.491137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1167/2360] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:19 2022] PUT /identity/v3/projects/6cb4220fc54b4afea0ef5bd16491a38b/users/eab92b5d6d2e4839a26836cd915b3aa7/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:19.495149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.505092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4914fa-d455-49ee-83e1-6da57b54d6a9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.562885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dc269e-4274-4680-a5cc-fe0c81d68d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.563560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc269e-4274-4680-a5cc-fe0c81d68d83 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.563945 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc269e-4274-4680-a5cc-fe0c81d68d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.564249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dc269e-4274-4680-a5cc-fe0c81d68d83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.565096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc269e-4274-4680-a5cc-fe0c81d68d83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.567066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dc269e-4274-4680-a5cc-fe0c81d68d83 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.578358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1194/2361] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:19.582398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.596352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19beb8-30f2-47da-bbc2-61fa59e8f620 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.600958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.601601 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.601873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.602142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] PATH_INFO: `/v3/projects/6cb4220fc54b4afea0ef5bd16491a38b/users/eab92b5d6d2e4839a26836cd915b3aa7/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.604020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cb4220fc54b4afea0ef5bd16491a38b, user_id=eab92b5d6d2e4839a26836cd915b3aa7, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.606830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.618543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1168/2362] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:19 2022] PUT /identity/v3/projects/6cb4220fc54b4afea0ef5bd16491a38b/users/eab92b5d6d2e4839a26836cd915b3aa7/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:19.629575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.629920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.630221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.657703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] MFA Rules not processed for user `eab92b5d6d2e4839a26836cd915b3aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:19.658931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:19.688254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:19.728394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1195/2363] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:19 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:19.740339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:19.798098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3743722-26fc-47a3-9618-a61468bac9b5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:19.802041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:19.802687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:19.803011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:19.803284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:19.812875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:19.814427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:19.823817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1169/2364] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:19 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:20.050552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:20.052171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.080781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:20.096933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da303ac-0b67-49b2-9a1c-2c75d2540358 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:20.101910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:20.104448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:20.105023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:20.105826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:20.109033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.134866 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d49a7bd-a21e-4a1c-af76-3989107f53b8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:20.145363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:20.146421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:20.147638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:20.148175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] PATH_INFO: `/v3/users/2e2b2801b3764d53ac36b59e23c98944` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:20.151010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2b2801b3764d53ac36b59e23c98944)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:20.153581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:20.196926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:20.198689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3d6f83-25c3-4f98-8977-eb32c1245259 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:20.211022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1170/2365] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:20.215258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2b2801b3764d53ac36b59e23c98944'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.215880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2b2801b3764d53ac36b59e23c98944'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.224360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2b2801b3764d53ac36b59e23c98944'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.225065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2b2801b3764d53ac36b59e23c98944'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.242490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2b2801b3764d53ac36b59e23c98944'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.258238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1196/2366] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:20 2022] DELETE /identity/v3/users/2e2b2801b3764d53ac36b59e23c98944 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:20.262437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:20.262608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:20.262894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:20.273058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:20.290249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:20.291786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.369360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1171/2367] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:20.380585 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:20.382513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.435858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:20.451113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd10ca2-99c9-4d0d-b106-b6709d9d5177 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:20.457602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:20.458400 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:20.458744 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:20.459028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:20.470699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:20.472358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:20.484514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1197/2368] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:20.511567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:20.514047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.601005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c999f1b-216e-4f17-bf04-608ef2a50f19 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:20.605909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:20.606969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:20.607420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:20.607837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:20.610682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.700312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:20.701862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:20.711472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1172/2369] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:20 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:20.798846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:20.802640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:20.825377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c47a8e-7c55-4cf5-8507-210de4013e4c service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:20.830689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:20.831311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:20.831586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:20.831858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] PATH_INFO: `/v3/projects/1d414e76296840fd965f67ff15ee67b0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:20.833496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d414e76296840fd965f67ff15ee67b0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:20.835617 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:20.893121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d414e76296840fd965f67ff15ee67b0', 'request_id': 'req-db370be3-3954-4cb4-9dd7-c28579722534'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.902511 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d414e76296840fd965f67ff15ee67b0', 'request_id': 'req-db370be3-3954-4cb4-9dd7-c28579722534'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.907987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d414e76296840fd965f67ff15ee67b0', 'request_id': 'req-db370be3-3954-4cb4-9dd7-c28579722534'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.923797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] The token cache is being invalidate because project 1d414e76296840fd965f67ff15ee67b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:20.924103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.924771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:20.930288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1198/2370] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:20 2022] DELETE /identity/v3/projects/1d414e76296840fd965f67ff15ee67b0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:20.931520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:20.959333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b259ee57-f9f2-4e70-9d0f-4e7102c624f4 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:21.030134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:21.031854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:21.061226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:21.062880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:21.062880 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:21.063079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] PATH_INFO: `/v3/users/f36146c2170b4f0d956f1008c0f8656f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:21.064572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f36146c2170b4f0d956f1008c0f8656f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:21.067159 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:21.069170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:21.120763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db370be3-3954-4cb4-9dd7-c28579722534 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:21.126331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:21.126959 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:21.127287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:21.127577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:21.129635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:21.150471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f36146c2170b4f0d956f1008c0f8656f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.152379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f36146c2170b4f0d956f1008c0f8656f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.163077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f36146c2170b4f0d956f1008c0f8656f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.163625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f36146c2170b4f0d956f1008c0f8656f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.178958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f36146c2170b4f0d956f1008c0f8656f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.199331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1173/2371] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:20 2022] DELETE /identity/v3/users/f36146c2170b4f0d956f1008c0f8656f => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:21.216496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:21.218794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:21.228686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1199/2372] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:21.576219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:21.578147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:21.652956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73246562-50ea-4fe7-953f-f7e1630a7db9 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:21.658387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:21.659032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:21.659306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:21.659579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] PATH_INFO: `/v3/projects/0bfd46ebccc44a249469c460271eec97` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:21.660805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bfd46ebccc44a249469c460271eec97)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:21.662710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:21.709438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bfd46ebccc44a249469c460271eec97', 'request_id': 'req-df5be873-afec-46ac-afbd-aec9a8808408'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.715119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bfd46ebccc44a249469c460271eec97', 'request_id': 'req-df5be873-afec-46ac-afbd-aec9a8808408'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.722935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bfd46ebccc44a249469c460271eec97', 'request_id': 'req-df5be873-afec-46ac-afbd-aec9a8808408'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.735501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] The token cache is being invalidate because project 0bfd46ebccc44a249469c460271eec97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:21.735790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.736673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.738193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1174/2373] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:21 2022] DELETE /identity/v3/projects/0bfd46ebccc44a249469c460271eec97 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:21.742372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:21.743807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:21.757429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d220475-df06-4930-904b-77ce2f58d8e2 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:21.774441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:21.817761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:21.818526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:21.818804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:21.819074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] PATH_INFO: `/v3/users/068da0fd2ef64d498554d69d533e6fc8` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:21.820199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068da0fd2ef64d498554d69d533e6fc8)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:21.822319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:21.868709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068da0fd2ef64d498554d69d533e6fc8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.869386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068da0fd2ef64d498554d69d533e6fc8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.877036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068da0fd2ef64d498554d69d533e6fc8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.877673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068da0fd2ef64d498554d69d533e6fc8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.891291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-06c18db5-bee9-4c3b-a1d3-31bdac060f09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068da0fd2ef64d498554d69d533e6fc8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:21.903480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1200/2374] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:21 2022] DELETE /identity/v3/users/068da0fd2ef64d498554d69d533e6fc8 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:22.175130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.176678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:22.205739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5be873-afec-46ac-afbd-aec9a8808408 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.211241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:22.266718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.267384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.267657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.267899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] PATH_INFO: `/v3/projects/83ed7d4705ed44a09b598fca8d7e8ba8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.269272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ed7d4705ed44a09b598fca8d7e8ba8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.271450 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.322598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ed7d4705ed44a09b598fca8d7e8ba8', 'request_id': 'req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:22.329851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ed7d4705ed44a09b598fca8d7e8ba8', 'request_id': 'req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:22.337544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ed7d4705ed44a09b598fca8d7e8ba8', 'request_id': 'req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:22.349118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] The token cache is being invalidate because project 83ed7d4705ed44a09b598fca8d7e8ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:22.349555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:22.350249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-a544fa44-353d-4ccf-be38-ff56f1fc1f6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:22.351732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1175/2375] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:22 2022] DELETE /identity/v3/projects/83ed7d4705ed44a09b598fca8d7e8ba8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:22.358175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.359207 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.359294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.379412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:22.381382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:22.442782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1201/2376] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:22.450295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.450581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.450863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.472290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:22.473493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:22.486315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1176/2377] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:22.487008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.497122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b22ffd-3b7a-4a39-bdb9-bef89293501c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.502339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.503015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.503289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.503555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.504199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:22.504492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.506368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.519100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1202/2378] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:22:22.523336 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.535065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975e094-e290-474e-ba4c-d63cb42e4d3b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.540489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.541089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.541364 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.541633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.542306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.544184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.580651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1177/2379] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:22 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:22.581909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.594133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbdd96-6f1d-4178-8180-712105046914 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.624446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:22.715395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.716029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.716311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.718562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.719814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.723310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.769224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1203/2380] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:22 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:22.774735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.787353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb150b6c-41be-4beb-a2a5-a909788f30ca admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.792945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.793546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.793892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.794319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.795009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.797420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.808224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1178/2381] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:22.812241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.822091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5da613-b8d6-47cc-8679-449cfd94ef92 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.826687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c96854a-669b-4f76-afea-de840fdeebc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.827792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96854a-669b-4f76-afea-de840fdeebc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.828066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96854a-669b-4f76-afea-de840fdeebc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.828335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c96854a-669b-4f76-afea-de840fdeebc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.828966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96854a-669b-4f76-afea-de840fdeebc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.830604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c96854a-669b-4f76-afea-de840fdeebc6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.840263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1204/2382] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:22.845342 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.855578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46ec56e-1604-4d26-81c3-9012dd987ce8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:22.860593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:22.861275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.861581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.861943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] PATH_INFO: `/v3/projects/acab4db636e64f4183798081133e0b67/users/4e47148f5c1e47b78a6a038329f0adc9/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.879077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acab4db636e64f4183798081133e0b67, user_id=4e47148f5c1e47b78a6a038329f0adc9, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:22.881700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:22.897530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1179/2383] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:22 2022] PUT /identity/v3/projects/acab4db636e64f4183798081133e0b67/users/4e47148f5c1e47b78a6a038329f0adc9/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:22.902489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f44c3a-3364-4127-a74d-0ed88ad034b7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:22.902848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f44c3a-3364-4127-a74d-0ed88ad034b7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:22.903184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f44c3a-3364-4127-a74d-0ed88ad034b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:22.922421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-55f44c3a-3364-4127-a74d-0ed88ad034b7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:22.923645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-55f44c3a-3364-4127-a74d-0ed88ad034b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:22.982305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1205/2384] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:22.992084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:22.993830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:23.033345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:23.051039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861d4a03-21dd-4dd7-89b3-3485d6ba3272 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:23.054568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:23.055153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:23.055427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:23.055692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:23.064496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:23.065889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:23.074951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1180/2385] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:23.734135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:23.734832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:23.735034 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:23.769592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] MFA Rules not processed for user `4e47148f5c1e47b78a6a038329f0adc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:23.771375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:23.852258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:23.855444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:23.870743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc693cd-788e-40a3-8b82-f81cc64d07cf service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:23.877088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:23.877892 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:23.878324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:23.878698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] PATH_INFO: `/v3/users/c13944d5f59245999657a257240dfa9b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:23.880207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c13944d5f59245999657a257240dfa9b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:23.882640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:23.987029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c13944d5f59245999657a257240dfa9b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:23.988948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c13944d5f59245999657a257240dfa9b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:23.999463 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c13944d5f59245999657a257240dfa9b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.000252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c13944d5f59245999657a257240dfa9b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.054576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1206/2386] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:23 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:24.054826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c0704efb-f6ca-4918-8088-8e77d462b516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c13944d5f59245999657a257240dfa9b'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.056852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:24.058732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:24.079925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1181/2387] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:23 2022] DELETE /identity/v3/users/c13944d5f59245999657a257240dfa9b => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:24.084395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:24.084803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:24.085344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:24.110934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:24.114472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:24.115955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:24.139838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a7638a-84de-4de0-bdeb-067a91321c1d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:24.221144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:24.222248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:24.222833 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:24.223499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:24.254462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1182/2388] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:24.272104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:24.274643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:24.312108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:24.316335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:24.330171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1207/2389] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:24.375686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac9465-4abe-477f-a556-9675105f8a08 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:24.381455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9f0ef3-0d96-4ea2-9d14-1e8f48283db8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:24.382749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0ef3-0d96-4ea2-9d14-1e8f48283db8 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:24.383184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0ef3-0d96-4ea2-9d14-1e8f48283db8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:24.383734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9f0ef3-0d96-4ea2-9d14-1e8f48283db8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:24.391533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:24.398543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f0ef3-0d96-4ea2-9d14-1e8f48283db8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:24.400368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9f0ef3-0d96-4ea2-9d14-1e8f48283db8 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:24.412276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1183/2390] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:24.786657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:24.789078 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:24.813882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9bfd0f-af44-4f45-a7ec-18d6c3d20223 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:24.823683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:24.824823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:24.824823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:24.825182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] PATH_INFO: `/v3/projects/7aa2df867ca54a0e9ae758e176c914b0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:24.826780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa2df867ca54a0e9ae758e176c914b0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:24.829358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:24.889271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa2df867ca54a0e9ae758e176c914b0', 'request_id': 'req-abd680cf-9251-4b2f-b199-ed7abaadbfa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.902727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa2df867ca54a0e9ae758e176c914b0', 'request_id': 'req-abd680cf-9251-4b2f-b199-ed7abaadbfa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.908932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa2df867ca54a0e9ae758e176c914b0', 'request_id': 'req-abd680cf-9251-4b2f-b199-ed7abaadbfa5'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.924783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] The token cache is being invalidate because project 7aa2df867ca54a0e9ae758e176c914b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:24.925072 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.925719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:24.927314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1208/2391] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:24 2022] DELETE /identity/v3/projects/7aa2df867ca54a0e9ae758e176c914b0 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:24.932219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb73a1-7561-47c4-87ff-69bd4abbcebc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:24.932691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb73a1-7561-47c4-87ff-69bd4abbcebc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:24.932878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb73a1-7561-47c4-87ff-69bd4abbcebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:24.957133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6fcb73a1-7561-47c4-87ff-69bd4abbcebc None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:24.958545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6fcb73a1-7561-47c4-87ff-69bd4abbcebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:24.972111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:24.973487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.007500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:25.032738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1184/2392] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:25.039008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd680cf-9251-4b2f-b199-ed7abaadbfa5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.041663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.041950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.042255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.042933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.043533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.043807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.044073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.046714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.066856 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:25.068081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.077645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1185/2393] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:25.082610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.093276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6472560-8596-43b4-b13a-d04f3aaf0802 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.098879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.099551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.099790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.100097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.100698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:25.101001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.102627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.115917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1186/2394] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:22:25.117041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.118586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.120047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.129331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1209/2395] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:25.130534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b9b66c-e707-406a-af0d-671f3f902ec0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.135350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.135951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.136232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.136506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.137129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.139299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.177304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1187/2396] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:25.178270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.193703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd43bd6-cd30-4a7a-af68-5e3805f471e4 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.260734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.261354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.261624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.261895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.262558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.265314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.292023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1210/2397] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:25.295963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.306279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08d3813-0d29-4ce3-a69a-ba1a299501cd admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.312293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.312854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.313147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.313408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.314087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.315884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.330114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1188/2398] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:25.331299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.341555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fe575-4b70-4495-8fea-2768645aa873 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.348296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95160c9-5118-4528-85eb-2e39d7cc281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.348915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95160c9-5118-4528-85eb-2e39d7cc281a admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.349196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95160c9-5118-4528-85eb-2e39d7cc281a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.349914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95160c9-5118-4528-85eb-2e39d7cc281a admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.350619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95160c9-5118-4528-85eb-2e39d7cc281a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.353027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95160c9-5118-4528-85eb-2e39d7cc281a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.365278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1211/2399] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:25.368993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.380260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73297a37-610a-4caf-a72a-e77278db85ef admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.385053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.385696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.385975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.386276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] PATH_INFO: `/v3/projects/67ef8b5346d545d9b66ce7c587468581/users/1c67721a3a8942518b83b45a7e79b8fd/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.402707 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] RBAC: Authorizing `identity:create_grant(project_id=67ef8b5346d545d9b66ce7c587468581, user_id=1c67721a3a8942518b83b45a7e79b8fd, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.405711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.430020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1189/2400] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:25 2022] PUT /identity/v3/projects/67ef8b5346d545d9b66ce7c587468581/users/1c67721a3a8942518b83b45a7e79b8fd/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:25.441405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.441839 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.442188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.478764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] MFA Rules not processed for user `1c67721a3a8942518b83b45a7e79b8fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:25.479991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.495642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.497766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.537896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:25.567544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1212/2401] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:25.568575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.575394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3417bae7-1148-47da-947f-e74ed2c1f37c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.580694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1116ee8-9bc1-4127-8af2-a3205fe1cb1a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.580824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.581496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.581785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.581935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] PATH_INFO: `/v3/users/499c907a7c5041db8456b97608869a96` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.583147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] RBAC: Authorizing `identity:delete_user(user_id=499c907a7c5041db8456b97608869a96)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.585054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.585321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.585949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.586241 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.586989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.587157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.589017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.611464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:25.634266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1213/2402] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/projects => generated 375 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:25.636403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.653681 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '499c907a7c5041db8456b97608869a96'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:25.654817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '499c907a7c5041db8456b97608869a96'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:25.663284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '499c907a7c5041db8456b97608869a96'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:25.664082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '499c907a7c5041db8456b97608869a96'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:25.682407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af559a24-bf8b-4694-a7fe-823090a17a89 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '499c907a7c5041db8456b97608869a96'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:25.694980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1190/2403] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:25 2022] DELETE /identity/v3/users/499c907a7c5041db8456b97608869a96 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:25.699351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.699742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.700079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.703125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e569563d-2360-4e6e-90cf-4572b2f9583d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.724150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:25.725378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.766637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.767935 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.768369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.768643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.769285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.771667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.803256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1214/2404] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/users => generated 294 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:25.807118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.808836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.821604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2377271-6187-4687-8456-ebea1f9c770b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.834076 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1191/2405] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:25.837238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.837853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.838515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.838515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.839148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.840785 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.846166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.847626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.853892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1215/2406] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:25 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:25.858352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:25.876453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236cd041-bdf3-4208-8ecb-da7ae2ebb59f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.881831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dcf848-381e-4ad9-aa8c-a8ed66dfb1c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.882849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcf848-381e-4ad9-aa8c-a8ed66dfb1c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.883129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcf848-381e-4ad9-aa8c-a8ed66dfb1c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.883398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dcf848-381e-4ad9-aa8c-a8ed66dfb1c9 admin admin] PATH_INFO: `/v3/projects/96b8b13caec14ccc866c3ca4ab90bd1e/users/a64f97ada277450d94a3f04eede64294/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.902733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcf848-381e-4ad9-aa8c-a8ed66dfb1c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b8b13caec14ccc866c3ca4ab90bd1e, user_id=a64f97ada277450d94a3f04eede64294, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:25.906807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dcf848-381e-4ad9-aa8c-a8ed66dfb1c9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:25.926749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1216/2407] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:25 2022] PUT /identity/v3/projects/96b8b13caec14ccc866c3ca4ab90bd1e/users/a64f97ada277450d94a3f04eede64294/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:25.928266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.928547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.928817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:25.942028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb06f3-a4d6-4cb9-95db-f7d7298b380a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:25.949568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] MFA Rules not processed for user `4e47148f5c1e47b78a6a038329f0adc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:25.950963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:25.982826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:25.995994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:25.996597 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:25.996917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:25.997217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:26.002996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:26.020268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1217/2408] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:25 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:26.022434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:26.069738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:26.071473 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:26.082459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1192/2409] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:26.103763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5825807f-60d7-46f9-8ef6-33406cb81d84 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:26.109156 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1ad10-540b-4039-828c-165dd22dbc7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:26.109769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1ad10-540b-4039-828c-165dd22dbc7e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:26.110044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1ad10-540b-4039-828c-165dd22dbc7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:26.110355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1ad10-540b-4039-828c-165dd22dbc7e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:26.111154 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1ad10-540b-4039-828c-165dd22dbc7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:26.113054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1ad10-540b-4039-828c-165dd22dbc7e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:26.125559 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1218/2410] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:26 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:26.129931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:26.141231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89986cdc-a27e-46ad-952a-15405946532f service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:26.146401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:26.147363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:26.147703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:26.148709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] PATH_INFO: `/v3/projects/96b8b13caec14ccc866c3ca4ab90bd1e/users/a64f97ada277450d94a3f04eede64294/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:26.151141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b8b13caec14ccc866c3ca4ab90bd1e, user_id=a64f97ada277450d94a3f04eede64294, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:26.154392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:26.167436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1193/2411] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:26 2022] PUT /identity/v3/projects/96b8b13caec14ccc866c3ca4ab90bd1e/users/a64f97ada277450d94a3f04eede64294/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:26.179119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:26.179657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:26.179910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:26.220518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] MFA Rules not processed for user `a64f97ada277450d94a3f04eede64294`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:26.224982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:26.347266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1219/2412] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:26 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:26.357587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:26.359609 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:26.426106 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6193be6d-4a99-42b6-95b3-0a333cf0e374 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:26.429846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:26.430483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:26.430762 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:26.431032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:26.432756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:26.491886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:26.493312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:26.503259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1194/2413] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:26 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:26.634146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:26.635527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:26.758755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:26.768604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a680cae-9098-4d6b-b45f-41e790865360 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:26.781553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:26.782236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:26.782515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:26.782789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] PATH_INFO: `/v3/projects/b80fc33076004182bcac3cd05dd7a262` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:26.785949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b80fc33076004182bcac3cd05dd7a262)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:26.787872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:26.937684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b80fc33076004182bcac3cd05dd7a262', 'request_id': 'req-e1024262-9bd1-47cf-a0e6-8d9161e6442d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:26.958173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b80fc33076004182bcac3cd05dd7a262', 'request_id': 'req-e1024262-9bd1-47cf-a0e6-8d9161e6442d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:26.965742 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b80fc33076004182bcac3cd05dd7a262', 'request_id': 'req-e1024262-9bd1-47cf-a0e6-8d9161e6442d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.001200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] The token cache is being invalidate because project b80fc33076004182bcac3cd05dd7a262 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:27.001651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.002675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.005894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1220/2414] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:26 2022] DELETE /identity/v3/projects/b80fc33076004182bcac3cd05dd7a262 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:27.010674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:27.033742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e0c1ab-b391-4e6c-b376-14071cbb45c2 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:27.130920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:27.164784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:27.165429 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:27.165711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:27.165985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] PATH_INFO: `/v3/users/e0e693d4933341edb5a8b322ce3eaa82` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:27.167452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0e693d4933341edb5a8b322ce3eaa82)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:27.171464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:27.274801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0e693d4933341edb5a8b322ce3eaa82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.275962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0e693d4933341edb5a8b322ce3eaa82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.299019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0e693d4933341edb5a8b322ce3eaa82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.300328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0e693d4933341edb5a8b322ce3eaa82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.317257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0e693d4933341edb5a8b322ce3eaa82'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.352793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1195/2415] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:27 2022] DELETE /identity/v3/users/e0e693d4933341edb5a8b322ce3eaa82 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:27.686080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:27.688696 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:27.710727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1024262-9bd1-47cf-a0e6-8d9161e6442d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:27.712170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:27.713557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:27.796445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee49e43d-eb8f-4dee-9fc0-8395017cf0c5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:27.800039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:27.800758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:27.801036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:27.801318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] PATH_INFO: `/v3/users/3a2ebc10e70843d3b25d997e95d424cb` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:27.802310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:27.802438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a2ebc10e70843d3b25d997e95d424cb)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:27.803071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:27.803348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:27.803959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] PATH_INFO: `/v3/projects/4774e509fb8b4e33b356b5246e8c6a9a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:27.804465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:27.804729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4774e509fb8b4e33b356b5246e8c6a9a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:27.806630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:27.866925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a2ebc10e70843d3b25d997e95d424cb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.868002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a2ebc10e70843d3b25d997e95d424cb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.880259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a2ebc10e70843d3b25d997e95d424cb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.881092 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a2ebc10e70843d3b25d997e95d424cb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.885174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4774e509fb8b4e33b356b5246e8c6a9a', 'request_id': 'req-13efe308-6ad9-48f3-812d-23905636b054'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.896044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-2bc54cd0-00d8-4f31-930e-5665c485602f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a2ebc10e70843d3b25d997e95d424cb'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.896231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4774e509fb8b4e33b356b5246e8c6a9a', 'request_id': 'req-13efe308-6ad9-48f3-812d-23905636b054'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.910474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4774e509fb8b4e33b356b5246e8c6a9a', 'request_id': 'req-13efe308-6ad9-48f3-812d-23905636b054'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.911909 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:27.917830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1221/2416] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:27 2022] DELETE /identity/v3/users/3a2ebc10e70843d3b25d997e95d424cb => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:27.925418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:27.925685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:27.926094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:27.927353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] The token cache is being invalidate because project 4774e509fb8b4e33b356b5246e8c6a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:27.927799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.928773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:27.930850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1196/2417] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:27 2022] DELETE /identity/v3/projects/4774e509fb8b4e33b356b5246e8c6a9a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:27.934981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:27.937177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:27.944571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:27.951046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:27.955416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:28.039024 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efe308-6ad9-48f3-812d-23905636b054 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:28.060108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1222/2418] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:28.072139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:28.075932 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:28.125632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:28.126301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:28.126582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:28.126943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] PATH_INFO: `/v3/users/20444801fc614006be1d69e7c62015f0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:28.128371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] RBAC: Authorizing `identity:delete_user(user_id=20444801fc614006be1d69e7c62015f0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:28.130448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:28.165712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc0148-237a-4ac3-a785-0d0c48793388 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:28.171057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:28.171870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:28.172151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:28.172419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:28.197370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20444801fc614006be1d69e7c62015f0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.198624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:28.200143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:28.200290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20444801fc614006be1d69e7c62015f0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.212297 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1223/2419] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:28.214872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20444801fc614006be1d69e7c62015f0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.215680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20444801fc614006be1d69e7c62015f0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.240793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20444801fc614006be1d69e7c62015f0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.275097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1197/2420] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:27 2022] DELETE /identity/v3/users/20444801fc614006be1d69e7c62015f0 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:28.606675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:28.609838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:28.628504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b41ed5-2c4f-4eb9-b28d-3b374e85c611 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:28.662230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:28.664967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:28.698058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:28.712257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:28.712896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:28.713193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:28.713468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] PATH_INFO: `/v3/projects/a2f367ccfb42458da4248644706b25e5` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:28.715103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2f367ccfb42458da4248644706b25e5)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:28.718167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:28.754857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05457945-feaf-48ea-b471-4432c0586002 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:28.759603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:28.760224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:28.760501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:28.760774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] PATH_INFO: `/v3/projects/71a387f8a0534afe9d9c8205f0a07ac3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:28.761862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a387f8a0534afe9d9c8205f0a07ac3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:28.763778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:28.776466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2f367ccfb42458da4248644706b25e5', 'request_id': 'req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.778800 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:28.784100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2f367ccfb42458da4248644706b25e5', 'request_id': 'req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.790644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2f367ccfb42458da4248644706b25e5', 'request_id': 'req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.805469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] The token cache is being invalidate because project a2f367ccfb42458da4248644706b25e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:28.805756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.806549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.808037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1224/2421] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:28 2022] DELETE /identity/v3/projects/a2f367ccfb42458da4248644706b25e5 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:28.812327 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:28.813627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:28.814591 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a387f8a0534afe9d9c8205f0a07ac3', 'request_id': 'req-3f9d25b8-8370-4021-906d-673baa4a5208'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.820772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a387f8a0534afe9d9c8205f0a07ac3', 'request_id': 'req-3f9d25b8-8370-4021-906d-673baa4a5208'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.827767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a387f8a0534afe9d9c8205f0a07ac3', 'request_id': 'req-3f9d25b8-8370-4021-906d-673baa4a5208'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.839187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] The token cache is being invalidate because project 71a387f8a0534afe9d9c8205f0a07ac3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:28.839479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.840116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:28.841583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1198/2422] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:28 2022] DELETE /identity/v3/projects/71a387f8a0534afe9d9c8205f0a07ac3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:28.887555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5459ef6e-0705-4d5e-b4f1-d6190e1aebb9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:28.949393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:28.950053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:28.950347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:28.950621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] PATH_INFO: `/v3/users/c9e53f3bd3124718aee1d42c203301f8` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:28.951961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9e53f3bd3124718aee1d42c203301f8)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:28.954014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:29.004821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9e53f3bd3124718aee1d42c203301f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.005521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9e53f3bd3124718aee1d42c203301f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.012705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9e53f3bd3124718aee1d42c203301f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.013435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9e53f3bd3124718aee1d42c203301f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.026115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9e53f3bd3124718aee1d42c203301f8'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.039107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1225/2423] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:28 2022] DELETE /identity/v3/users/c9e53f3bd3124718aee1d42c203301f8 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:29.291942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:29.293704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:29.308382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d25b8-8370-4021-906d-673baa4a5208 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:29.315872 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:29.317622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:29.328217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:29.394856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed368fd3-ae7d-4b06-bee6-b28767a7bc36 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:29.400794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:29.401451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:29.401906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:29.402180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] PATH_INFO: `/v3/projects/817c79d7a4c749a18ab265a45257ee19` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:29.403367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] RBAC: Authorizing `identity:delete_project(project_id=817c79d7a4c749a18ab265a45257ee19)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:29.406096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:29.410842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:29.411474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:29.411751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:29.412025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] PATH_INFO: `/v3/users/1c67721a3a8942518b83b45a7e79b8fd` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:29.413525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c67721a3a8942518b83b45a7e79b8fd)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:29.415446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:29.461633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817c79d7a4c749a18ab265a45257ee19', 'request_id': 'req-08cddaca-c23a-4929-9da7-5d94e874c59d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.469625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c67721a3a8942518b83b45a7e79b8fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.469949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817c79d7a4c749a18ab265a45257ee19', 'request_id': 'req-08cddaca-c23a-4929-9da7-5d94e874c59d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.470318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c67721a3a8942518b83b45a7e79b8fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.475196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:29.476093 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817c79d7a4c749a18ab265a45257ee19', 'request_id': 'req-08cddaca-c23a-4929-9da7-5d94e874c59d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.479339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c67721a3a8942518b83b45a7e79b8fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.480355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c67721a3a8942518b83b45a7e79b8fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.490466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] The token cache is being invalidate because project 817c79d7a4c749a18ab265a45257ee19 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:29.490761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.491982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.494246 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1226/2424] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:29 2022] DELETE /identity/v3/projects/817c79d7a4c749a18ab265a45257ee19 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:29.497323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:29.498791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:29.498915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-cd10b240-b709-4ac8-954b-a8917a3f61db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c67721a3a8942518b83b45a7e79b8fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.516207 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1199/2425] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:29 2022] DELETE /identity/v3/users/1c67721a3a8942518b83b45a7e79b8fd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:29.520424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:29.520716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:29.520999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:29.549486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:29.551014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:29.574781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cddaca-c23a-4929-9da7-5d94e874c59d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:29.623412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:29.638230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1200/2426] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:29.645201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:29.646464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:29.646629 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:29.647107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:29.647386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:29.647662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] PATH_INFO: `/v3/users/eab92b5d6d2e4839a26836cd915b3aa7` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:29.648916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eab92b5d6d2e4839a26836cd915b3aa7)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:29.651047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:29.706164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eab92b5d6d2e4839a26836cd915b3aa7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.706880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eab92b5d6d2e4839a26836cd915b3aa7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.710794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5faa9ce4-c183-4cc7-9ed9-2e06938e6a61 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:29.714553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:29.714882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eab92b5d6d2e4839a26836cd915b3aa7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.715083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:29.715349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:29.715581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eab92b5d6d2e4839a26836cd915b3aa7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.715688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:29.717547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:29.728922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eab92b5d6d2e4839a26836cd915b3aa7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:29.730110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:29.731550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:29.742808 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1227/2427] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:29 2022] DELETE /identity/v3/users/eab92b5d6d2e4839a26836cd915b3aa7 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:29.743329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1201/2428] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:30.113009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.114724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:30.202048 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45910447-390b-418e-a47b-c3445f7424e1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.206639 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.207260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.207533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.207796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] PATH_INFO: `/v3/projects/67ef8b5346d545d9b66ce7c587468581` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.208896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67ef8b5346d545d9b66ce7c587468581)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.210827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.263041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67ef8b5346d545d9b66ce7c587468581', 'request_id': 'req-692d9f75-4869-4cca-8c16-af5bbffe9a15'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.271894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67ef8b5346d545d9b66ce7c587468581', 'request_id': 'req-692d9f75-4869-4cca-8c16-af5bbffe9a15'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.280878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67ef8b5346d545d9b66ce7c587468581', 'request_id': 'req-692d9f75-4869-4cca-8c16-af5bbffe9a15'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.281040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.283502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:30.284129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:30.302044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] The token cache is being invalidate because project 67ef8b5346d545d9b66ce7c587468581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:30.302389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.303022 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38777c5e-2317-4e0e-954b-d19bed2af8db service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.303504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.304869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1228/2429] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:30 2022] DELETE /identity/v3/projects/67ef8b5346d545d9b66ce7c587468581 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:30.308775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.310150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:30.402679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.403727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.404223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.404695 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] PATH_INFO: `/v3/projects/6cb4220fc54b4afea0ef5bd16491a38b` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.407767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cb4220fc54b4afea0ef5bd16491a38b)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.410271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.431938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:30.437755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d9f75-4869-4cca-8c16-af5bbffe9a15 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.448336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.448970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.449471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.449840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] PATH_INFO: `/v3/users/a64f97ada277450d94a3f04eede64294` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.451297 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a64f97ada277450d94a3f04eede64294)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.454377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.507896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cb4220fc54b4afea0ef5bd16491a38b', 'request_id': 'req-fffdd33c-c191-4689-9b84-ae4df114ae72'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.514769 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cb4220fc54b4afea0ef5bd16491a38b', 'request_id': 'req-fffdd33c-c191-4689-9b84-ae4df114ae72'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.520646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a64f97ada277450d94a3f04eede64294'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.521626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a64f97ada277450d94a3f04eede64294'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.523911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cb4220fc54b4afea0ef5bd16491a38b', 'request_id': 'req-fffdd33c-c191-4689-9b84-ae4df114ae72'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.529970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a64f97ada277450d94a3f04eede64294'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.530880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a64f97ada277450d94a3f04eede64294'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.537516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] The token cache is being invalidate because project 6cb4220fc54b4afea0ef5bd16491a38b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:30.538356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.538471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-fffdd33c-c191-4689-9b84-ae4df114ae72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.540456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1202/2430] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:30 2022] DELETE /identity/v3/projects/6cb4220fc54b4afea0ef5bd16491a38b => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:30.544772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-efe1afd5-1387-4050-aad7-397f20159389 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a64f97ada277450d94a3f04eede64294'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:30.548046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.548603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.548885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.563317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1229/2431] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:30 2022] DELETE /identity/v3/users/a64f97ada277450d94a3f04eede64294 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:30.574981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:30.576764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:30.648536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1203/2432] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:30.658753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.659586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.659929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.687457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:30.688743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:30.704192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1230/2433] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:30.705751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.719863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6bce4b-9d83-4855-be56-d1edee492904 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.727758 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.729046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.729659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.730135 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.731687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:30.732223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.734755 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.753945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1204/2434] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:22:30.758437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.770524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d667eb9-aa1a-4810-a6b9-50f8c9c19cb8 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.776128 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.776977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.777090 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.777438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.778220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.781629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.815270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1231/2435] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:30 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:30.819183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.831026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcc9cd-3fdd-4763-866e-eff966f184d3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.900047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.900640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.900982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.901256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.901893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.906857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.907898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.909721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:30.929509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8397a68-6998-4e99-bf1f-51581ff079de admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.932125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1205/2436] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:30 2022] POST /identity/v3/users => generated 291 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:30.936345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.938215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.939603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.940229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.940586 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] PATH_INFO: `/v3/projects/96b8b13caec14ccc866c3ca4ab90bd1e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.941810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b8b13caec14ccc866c3ca4ab90bd1e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.943820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.950448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5200fd-78c3-4461-96f0-4858ba397601 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:30.956702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:30.957753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:30.958040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:30.958420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:30.959205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:30.961253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:30.978817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1206/2437] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:30.980222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:30.990924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:30.994491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc2b68d-1b8a-4873-b69c-03b42142bbd3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.001623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:31.002451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b8b13caec14ccc866c3ca4ab90bd1e', 'request_id': 'req-4f187460-4d9a-4761-8d7d-a48d5a2903d3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:31.011370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b8b13caec14ccc866c3ca4ab90bd1e', 'request_id': 'req-4f187460-4d9a-4761-8d7d-a48d5a2903d3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:31.018631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b8b13caec14ccc866c3ca4ab90bd1e', 'request_id': 'req-4f187460-4d9a-4761-8d7d-a48d5a2903d3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:31.036804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] The token cache is being invalidate because project 96b8b13caec14ccc866c3ca4ab90bd1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:31.037139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:31.037736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-4f187460-4d9a-4761-8d7d-a48d5a2903d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:31.039620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1232/2438] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:30 2022] DELETE /identity/v3/projects/96b8b13caec14ccc866c3ca4ab90bd1e => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:31.044094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a87573-d308-4781-a9f4-0113b4a8ccd6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.044381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a87573-d308-4781-a9f4-0113b4a8ccd6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.044678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a87573-d308-4781-a9f4-0113b4a8ccd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.067334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.067974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.068279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.068553 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.069248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.070927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.071044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-81a87573-d308-4781-a9f4-0113b4a8ccd6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:31.072381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-81a87573-d308-4781-a9f4-0113b4a8ccd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.081518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1233/2439] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:31.082776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1207/2440] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:30 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:31.090442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.090886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.091915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.092335 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.093452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.110994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa9df13-7441-4d18-8c23-570f0296f420 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.118040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d17fb5-d086-4fac-a628-40a3b9d4c52e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.120127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17fb5-d086-4fac-a628-40a3b9d4c52e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.120520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17fb5-d086-4fac-a628-40a3b9d4c52e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.120861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d17fb5-d086-4fac-a628-40a3b9d4c52e admin admin] PATH_INFO: `/v3/projects/9e814fc8005d4941b5eec7aa06144683/users/a7a1fdb5bd22468c99a0024d24f6d606/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.126501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:31.128274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.141676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1234/2441] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:31.144881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17fb5-d086-4fac-a628-40a3b9d4c52e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e814fc8005d4941b5eec7aa06144683, user_id=a7a1fdb5bd22468c99a0024d24f6d606, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.146107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.152701 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d17fb5-d086-4fac-a628-40a3b9d4c52e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.165188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6153db-3347-40a8-9f62-60ca46486fa1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.179241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1208/2442] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:31 2022] PUT /identity/v3/projects/9e814fc8005d4941b5eec7aa06144683/users/a7a1fdb5bd22468c99a0024d24f6d606/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:31.180755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.181375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.181659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.181921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.182629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:31.183065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.184570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.184729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.184904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.185114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.205677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1235/2443] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:22:31.207940 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.212555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:31.213761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.220612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5640c84-6b96-48a3-a481-9c0eb9342af5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.289639 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1209/2444] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:31.299854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.300524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.300841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.301123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.301766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.304018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.304510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.305558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.335841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1236/2445] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:31 2022] POST /identity/v3/projects => generated 373 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:31.340145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.373451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa489f6c-f1ba-4ddd-8d92-23d22e143ec2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.417408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:31.423745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91926a-d590-4bb4-91f5-c0351f8932d4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.429899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.430937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.431060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.431444 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.432180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.434515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.438698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.439308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.439582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.439852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.449956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.451487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.461622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1237/2446] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:31 2022] POST /identity/v3/users => generated 287 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:31.466721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.471550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1210/2447] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:31.479539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb1e2a-e7af-472e-8fb1-719fc8cbe63a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.486288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.487014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.487356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.488172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.488391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.490577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.508416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1238/2448] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:31.510155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.523396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd84f20-8290-4639-a1b8-3b1eeda0c8f0 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.531110 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ce6894-8796-4756-833c-d48ac6aa407d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.531793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce6894-8796-4756-833c-d48ac6aa407d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.532143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce6894-8796-4756-833c-d48ac6aa407d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.532434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ce6894-8796-4756-833c-d48ac6aa407d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.533294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce6894-8796-4756-833c-d48ac6aa407d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.534686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ce6894-8796-4756-833c-d48ac6aa407d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.545655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1211/2449] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:22:31.549661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.557117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:31.563740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff248313-100d-4f45-82bb-cb7ad2568e2c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.571199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.571985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.572198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.572465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] PATH_INFO: `/v3/projects/322f5025b631410aa54e01292614a1b7/users/7f2698581d6a4471936d27d210606ee6/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.592988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] RBAC: Authorizing `identity:create_grant(project_id=322f5025b631410aa54e01292614a1b7, user_id=7f2698581d6a4471936d27d210606ee6, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.597065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.615377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1239/2450] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:31 2022] PUT /identity/v3/projects/322f5025b631410aa54e01292614a1b7/users/7f2698581d6a4471936d27d210606ee6/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:22:31.628057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1340cd-b3ae-4f9a-b476-ec28d239dac0 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.629548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1340cd-b3ae-4f9a-b476-ec28d239dac0 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.629994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1340cd-b3ae-4f9a-b476-ec28d239dac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.665559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ce1340cd-b3ae-4f9a-b476-ec28d239dac0 None None] MFA Rules not processed for user `7f2698581d6a4471936d27d210606ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:31.667257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ce1340cd-b3ae-4f9a-b476-ec28d239dac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.770474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1212/2451] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:31.782854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:31.784834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:31.856488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e6ab74-67ec-4716-8a97-3764b32b885f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:31.861002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:31.861845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:31.862067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:31.862364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:31.877514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:31.879337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:31.890385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1240/2452] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:32.656513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e53241-87c1-45b1-893a-15a02c5fa79c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:32.656973 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e53241-87c1-45b1-893a-15a02c5fa79c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:32.657092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e53241-87c1-45b1-893a-15a02c5fa79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:32.684917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-26e53241-87c1-45b1-893a-15a02c5fa79c None None] MFA Rules not processed for user `a7a1fdb5bd22468c99a0024d24f6d606`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:32.686115 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-26e53241-87c1-45b1-893a-15a02c5fa79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:32.769169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1213/2453] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:32 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:32.785826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:32.797922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c26a6b-f3fa-4924-874f-af10b3929e76 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:32.801994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527fbd0a-6feb-4f45-9048-43026724b7d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:32.802687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fbd0a-6feb-4f45-9048-43026724b7d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:32.802968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fbd0a-6feb-4f45-9048-43026724b7d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:32.803250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527fbd0a-6feb-4f45-9048-43026724b7d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:32.813964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fbd0a-6feb-4f45-9048-43026724b7d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:32.815367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527fbd0a-6feb-4f45-9048-43026724b7d9 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:32.826840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1241/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:32 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:22:34.288030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d119f-4a9f-46e9-a948-50eb8063f9a4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:34.288514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d119f-4a9f-46e9-a948-50eb8063f9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:34.288593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4d119f-4a9f-46e9-a948-50eb8063f9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:34.297544 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:34.298022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:34.298151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:34.308354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2c4d119f-4a9f-46e9-a948-50eb8063f9a4 None None] MFA Rules not processed for user `28c104b3e1284882b72d6b6933475029`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:34.309619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2c4d119f-4a9f-46e9-a948-50eb8063f9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:34.317009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] MFA Rules not processed for user `a7a1fdb5bd22468c99a0024d24f6d606`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:34.318279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:34.325551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1242/2455] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:22:34.361730 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:34.378289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1214/2456] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:34 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:42.683336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:42.743519 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac1315-9159-4893-8081-74c307d1e6e1 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:42.747910 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:42.749018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:42.749432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:42.749703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] PATH_INFO: `/v3/users/7f2698581d6a4471936d27d210606ee6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:42.750799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f2698581d6a4471936d27d210606ee6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:42.752699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:42.797355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f2698581d6a4471936d27d210606ee6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:42.798006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f2698581d6a4471936d27d210606ee6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:42.805058 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f2698581d6a4471936d27d210606ee6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:42.805681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f2698581d6a4471936d27d210606ee6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:42.813083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:INSERT=1,keystone:DELETE=8 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:42.819266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f2698581d6a4471936d27d210606ee6'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:42.832393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1243/2457] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:22:42 2022] DELETE /identity/v3/users/7f2698581d6a4471936d27d210606ee6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:42.836737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:42.837215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:42.837326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:42.856416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:42.857696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:42.916043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1215/2458] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:42.925824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:42.927255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:42.984750 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a399f4-a8b6-4f7e-b7eb-166e39e102a9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:42.988323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6685e-eb89-49f2-8f2a-de20c578dce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:42.988922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6685e-eb89-49f2-8f2a-de20c578dce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:42.989196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6685e-eb89-49f2-8f2a-de20c578dce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:42.989466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6685e-eb89-49f2-8f2a-de20c578dce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:42.999272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6685e-eb89-49f2-8f2a-de20c578dce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.000766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6685e-eb89-49f2-8f2a-de20c578dce9 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.010709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1244/2459] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:43.286260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.287852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:43.302792 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd32c9-858b-47ab-9d24-5c8bfd983e78 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.308145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.308819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.309103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.309370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] PATH_INFO: `/v3/projects/322f5025b631410aa54e01292614a1b7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.310555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=322f5025b631410aa54e01292614a1b7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.313499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.365465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '322f5025b631410aa54e01292614a1b7', 'request_id': 'req-159feb49-99de-43bb-987a-a7bd3cb516b5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:43.371945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '322f5025b631410aa54e01292614a1b7', 'request_id': 'req-159feb49-99de-43bb-987a-a7bd3cb516b5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:43.381657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '322f5025b631410aa54e01292614a1b7', 'request_id': 'req-159feb49-99de-43bb-987a-a7bd3cb516b5'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:43.394704 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] The token cache is being invalidate because project 322f5025b631410aa54e01292614a1b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:22:43.395175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:43.395929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-159feb49-99de-43bb-987a-a7bd3cb516b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:22:43.399935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1216/2460] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:22:43 2022] DELETE /identity/v3/projects/322f5025b631410aa54e01292614a1b7 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:43.404797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.405137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.405412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.425780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:43.427427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:43.492649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1245/2461] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:43.501082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.501543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.501953 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.523535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:43.524784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:43.533382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1217/2462] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:43.537816 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.548908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498af193-bcec-4afd-8d7c-7167e93c0b39 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.553690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.554357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.554665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.554942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.555593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:22:43.555895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.557521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.583406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1246/2463] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:22:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:22:43.584854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.595603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc24d2a7-95f9-4e5e-b015-c1cffc53397f None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.600226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.600819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.601096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.601394 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.602029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.603972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.644282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1218/2464] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:22:43 2022] POST /identity/v3/projects => generated 391 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:22:43.645431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.656989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f633ba9-1505-4f08-bdaa-b0b601369b0e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.712722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.713394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.713588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.713859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.714560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.716507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.740542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1247/2465] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:22:43 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:22:43.744819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.755322 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4042eab8-66c6-469b-885f-6b586f5f798f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.760124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.760799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.761139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.761497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.762273 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.763996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.774285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1219/2466] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:43.778237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.784200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:43.790394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e41a4e7-5ba3-4e43-aaeb-5ebe11418ad6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.797642 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa07398-6279-4ea7-887e-7bd2cb2cf1f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.798473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa07398-6279-4ea7-887e-7bd2cb2cf1f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.798893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa07398-6279-4ea7-887e-7bd2cb2cf1f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.799474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa07398-6279-4ea7-887e-7bd2cb2cf1f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.800504 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa07398-6279-4ea7-887e-7bd2cb2cf1f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.803387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa07398-6279-4ea7-887e-7bd2cb2cf1f8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.815568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1248/2467] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:22:43.819825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:43.830651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd899c-5cf5-4b31-8fc5-baead720d6c4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:43.835345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:43.835997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.836275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.836546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] PATH_INFO: `/v3/projects/1e8c07c561e94b438e892d16c57c0e15/users/fe310ac41f4043f1a0540fc60e98099a/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.853392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e8c07c561e94b438e892d16c57c0e15, user_id=fe310ac41f4043f1a0540fc60e98099a, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:43.856061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:43.871197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1220/2468] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:22:43 2022] PUT /identity/v3/projects/1e8c07c561e94b438e892d16c57c0e15/users/fe310ac41f4043f1a0540fc60e98099a/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:22:43.882151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b220c-9e1b-492e-92bc-f70df427a691 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:43.882468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b220c-9e1b-492e-92bc-f70df427a691 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:43.882740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b220c-9e1b-492e-92bc-f70df427a691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:43.912135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d79b220c-9e1b-492e-92bc-f70df427a691 None None] MFA Rules not processed for user `fe310ac41f4043f1a0540fc60e98099a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:43.913658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d79b220c-9e1b-492e-92bc-f70df427a691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:43.987465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1249/2469] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:43.999202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:22:44.001214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:44.015648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:22:44.065517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faf2a65-f099-4199-91d9-160b4284ffd4 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:22:44.069375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:22:44.069984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:44.070288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:44.070563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:44.079307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:22:44.080729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:22:44.089689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1221/2470] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:22:43 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:22:53.620566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b57ec-8fcf-486b-91e9-b567fd8905f4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:22:53.621046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b57ec-8fcf-486b-91e9-b567fd8905f4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:22:53.621139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b57ec-8fcf-486b-91e9-b567fd8905f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:22:53.640629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4c7b57ec-8fcf-486b-91e9-b567fd8905f4 None None] MFA Rules not processed for user `4e47148f5c1e47b78a6a038329f0adc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:22:53.641836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4c7b57ec-8fcf-486b-91e9-b567fd8905f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:22:53.695419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1250/2471] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:22:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:22:54.085551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=20 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:02.532064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:02.606733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be87038-3050-4547-bc55-ca7637ccb420 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:02.611938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:02.612517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:02.612798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:02.613072 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] PATH_INFO: `/v3/users/fe310ac41f4043f1a0540fc60e98099a` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:02.614183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe310ac41f4043f1a0540fc60e98099a)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:02.620634 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:02.691776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe310ac41f4043f1a0540fc60e98099a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:02.692474 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe310ac41f4043f1a0540fc60e98099a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:02.699765 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe310ac41f4043f1a0540fc60e98099a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:02.700408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe310ac41f4043f1a0540fc60e98099a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:02.735038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe310ac41f4043f1a0540fc60e98099a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:02.754870 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1222/2472] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:23:02 2022] DELETE /identity/v3/users/fe310ac41f4043f1a0540fc60e98099a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:02.759184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:02.759637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:02.759853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:02.780964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:02.782185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:02.843861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1251/2473] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:02.854370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:02.856212 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:02.916266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1434d-61f0-4895-884e-d905245bbad8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:02.920984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80dad51-0136-4d10-8220-2f9346d4ca33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:02.921732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dad51-0136-4d10-8220-2f9346d4ca33 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:02.922013 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dad51-0136-4d10-8220-2f9346d4ca33 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:02.922439 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80dad51-0136-4d10-8220-2f9346d4ca33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:02.932949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dad51-0136-4d10-8220-2f9346d4ca33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:02.934413 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80dad51-0136-4d10-8220-2f9346d4ca33 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:02.944500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1223/2474] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:23:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:23:03.233897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.235514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:03.249537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e259ab-19b2-403f-9537-30ae2482162e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.256199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.256874 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.257146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.257413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] PATH_INFO: `/v3/projects/1e8c07c561e94b438e892d16c57c0e15` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.261859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e8c07c561e94b438e892d16c57c0e15)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.264669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.289591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:03.317826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e8c07c561e94b438e892d16c57c0e15', 'request_id': 'req-cb84625f-1196-41df-921e-6e0f3183cc79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:03.325006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e8c07c561e94b438e892d16c57c0e15', 'request_id': 'req-cb84625f-1196-41df-921e-6e0f3183cc79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:03.334066 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e8c07c561e94b438e892d16c57c0e15', 'request_id': 'req-cb84625f-1196-41df-921e-6e0f3183cc79'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:03.349825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] The token cache is being invalidate because project 1e8c07c561e94b438e892d16c57c0e15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:23:03.350507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:03.351913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cb84625f-1196-41df-921e-6e0f3183cc79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:03.360779 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1252/2475] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:23:03 2022] DELETE /identity/v3/projects/1e8c07c561e94b438e892d16c57c0e15 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:03.362675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.363149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.363496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.392285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:03.393514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:03.413490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:03.472549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1224/2476] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:03.479445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.479724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.479992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.505190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:03.506480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:03.514554 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1253/2477] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:03.518976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.528462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e07284d-472e-4ce4-9e04-9fa81d45f18d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.533065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.533670 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.533920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.534186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.534847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:23:03.535143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.536719 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.550496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1225/2478] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:23:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:23:03.554665 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.565147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc5a7b2-43e9-4217-a4b1-f365ec21815e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.571395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.572068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.572414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.572760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.573457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.575900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.605882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1254/2479] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:23:03 2022] POST /identity/v3/projects => generated 385 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:23:03.610462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.625141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c64416-be83-405c-8838-5a4795c1a376 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.693054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.693666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.693939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.694238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.695030 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.696957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.728226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1226/2480] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:23:03 2022] POST /identity/v3/users => generated 293 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:23:03.733251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.751946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c520fa3-034b-4ed9-b960-d8a6647760b5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.757853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.759302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.759302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.760051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.760389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.766299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.775130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1255/2481] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:23:03.779469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.798917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10914635-c8b4-46d2-89a8-ada777f9ac98 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.816551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6a97e-4455-4226-98e9-ae09caba0d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.817312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6a97e-4455-4226-98e9-ae09caba0d1f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.817620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6a97e-4455-4226-98e9-ae09caba0d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.817918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6a97e-4455-4226-98e9-ae09caba0d1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.818600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6a97e-4455-4226-98e9-ae09caba0d1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.820280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6a97e-4455-4226-98e9-ae09caba0d1f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.850164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1227/2482] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:23:03 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:23:03.854063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:03.866131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6469de76-ffab-4acd-b31e-70f282252def admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:03.871916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:03.872996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.873378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.874435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] PATH_INFO: `/v3/projects/c3a2d556879e4a57ae3e889536468a8c/users/7ed8b85b47694218add98b6ed986eff3/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.893038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a2d556879e4a57ae3e889536468a8c, user_id=7ed8b85b47694218add98b6ed986eff3, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:03.896108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:03.947221 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1256/2483] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:23:03 2022] PUT /identity/v3/projects/c3a2d556879e4a57ae3e889536468a8c/users/7ed8b85b47694218add98b6ed986eff3/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:23:03.953219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42019dc1-7a51-497e-b4c6-b64315bad2b8 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:03.953539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42019dc1-7a51-497e-b4c6-b64315bad2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:03.953867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42019dc1-7a51-497e-b4c6-b64315bad2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:03.993800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-42019dc1-7a51-497e-b4c6-b64315bad2b8 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:03.995099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-42019dc1-7a51-497e-b4c6-b64315bad2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:04.142714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1228/2484] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:04.151941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:04.154587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:04.228198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bb51b2-5368-41c4-a7d8-3539de9984f8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:04.232590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:04.233121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:04.233394 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:04.233661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:04.250629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:04.252884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:04.265634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1257/2485] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:23:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:23:08.780459 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:08.781003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:08.781003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:08.813754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] MFA Rules not processed for user `7ed8b85b47694218add98b6ed986eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:08.814899 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:08.834585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:08.895351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1229/2486] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:08.899925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:08.909984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8fcf6-dcae-4bef-b145-4ca899423eda service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:08.914525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:08.915106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:08.915379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:08.915644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:08.916258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:08.918132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:08.955829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1258/2487] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:23:08 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:23:08.957018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:08.968160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917a48db-188b-4ff0-8d31-3fdf3d298bbb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:09.037299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:09.037906 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:09.038182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:09.038488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:09.039117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:09.041037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:09.063570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1230/2488] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:23:08 2022] POST /identity/v3/users => generated 299 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:23:09.067445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:09.077834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175d6207-95af-4e31-8354-66fa44bd628f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:09.082395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:09.083027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:09.083323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:09.083594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:09.084232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:09.085814 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:09.102524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1259/2489] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:23:09.104804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:09.117636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf87f93-49a6-45a3-b5e5-eded880b41d2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:09.124750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:09.125716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:09.126232 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:09.126710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] PATH_INFO: `/v3/projects/7970519189e04aa7afc0e66511ae49d0/users/fbcaaf3b79ce4ca59d74081eeb6a5091/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:09.146074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7970519189e04aa7afc0e66511ae49d0, user_id=fbcaaf3b79ce4ca59d74081eeb6a5091, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:09.149534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:09.176731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1231/2490] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:23:09 2022] PUT /identity/v3/projects/7970519189e04aa7afc0e66511ae49d0/users/fbcaaf3b79ce4ca59d74081eeb6a5091/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:09.181166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:09.194390 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c209889-83ef-4110-9ad6-65e37526b224 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:09.208876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:09.277757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208e3de-d543-4eac-ab4b-104820da1ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:09.278122 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208e3de-d543-4eac-ab4b-104820da1ee5 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:09.278972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208e3de-d543-4eac-ab4b-104820da1ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:09.279516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208e3de-d543-4eac-ab4b-104820da1ee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:09.280553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208e3de-d543-4eac-ab4b-104820da1ee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:09.282352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208e3de-d543-4eac-ab4b-104820da1ee5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:09.294873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1260/2491] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:23:09.299029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:09.311920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c62a0a-12de-4162-8f3c-4125d9cb0f38 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:09.318031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:09.318806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:09.319911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:09.320312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] PATH_INFO: `/v3/projects/7970519189e04aa7afc0e66511ae49d0/users/fbcaaf3b79ce4ca59d74081eeb6a5091/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:09.323091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7970519189e04aa7afc0e66511ae49d0, user_id=fbcaaf3b79ce4ca59d74081eeb6a5091, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:09.326479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:09.339021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1232/2492] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:23:09 2022] PUT /identity/v3/projects/7970519189e04aa7afc0e66511ae49d0/users/fbcaaf3b79ce4ca59d74081eeb6a5091/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:13.992525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b7c55-4ad8-4af3-95a2-8f1a28055a87 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:13.993013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b7c55-4ad8-4af3-95a2-8f1a28055a87 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:13.993113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0b7c55-4ad8-4af3-95a2-8f1a28055a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:14.032293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3d0b7c55-4ad8-4af3-95a2-8f1a28055a87 None None] MFA Rules not processed for user `fbcaaf3b79ce4ca59d74081eeb6a5091`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:14.033889 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3d0b7c55-4ad8-4af3-95a2-8f1a28055a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:14.178912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1261/2493] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:13 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:14.238966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:14.241377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:14.346104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b0c560-61c3-48fd-9413-10f5d34d8972 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:14.350176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22342430-942b-4024-8d17-900365967ae8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:14.350921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22342430-942b-4024-8d17-900365967ae8 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:14.351268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22342430-942b-4024-8d17-900365967ae8 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:14.351607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22342430-942b-4024-8d17-900365967ae8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:14.356227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:14.446142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22342430-942b-4024-8d17-900365967ae8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:14.448542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22342430-942b-4024-8d17-900365967ae8 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:14.463235 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1233/2494] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:23:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:23:14.672085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8f77-8f69-492c-9981-67a7ab44c674 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:14.672366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8f77-8f69-492c-9981-67a7ab44c674 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:14.672635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7a8f77-8f69-492c-9981-67a7ab44c674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:14.696933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-0b7a8f77-8f69-492c-9981-67a7ab44c674 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:14.698179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0b7a8f77-8f69-492c-9981-67a7ab44c674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:14.761850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1262/2495] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:23:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:14.820193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22342430-942b-4024-8d17-900365967ae8 service glance] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:14.821799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-22342430-942b-4024-8d17-900365967ae8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:14.912834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22342430-942b-4024-8d17-900365967ae8 service glance] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:14.917754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0365fc2-c84a-425b-acda-240745f1e420 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:14.918398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0365fc2-c84a-425b-acda-240745f1e420 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:14.918678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0365fc2-c84a-425b-acda-240745f1e420 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:14.918998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0365fc2-c84a-425b-acda-240745f1e420 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:14.926844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0365fc2-c84a-425b-acda-240745f1e420 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:14.928400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0365fc2-c84a-425b-acda-240745f1e420 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:14.937118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1234/2496] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:23:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:23:24.754764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=81 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:24.829983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:24.830424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:24.830780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:24.852233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] MFA Rules not processed for user `7ed8b85b47694218add98b6ed986eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:24.854534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:24.862234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1263/2497] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:24.863600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94544012-49d8-48a7-8210-6baee9bb5193 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:24.864086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94544012-49d8-48a7-8210-6baee9bb5193 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:24.864403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94544012-49d8-48a7-8210-6baee9bb5193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:24.886524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-94544012-49d8-48a7-8210-6baee9bb5193 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:24.888401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-94544012-49d8-48a7-8210-6baee9bb5193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:24.895872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1235/2498] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:23:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:24.909298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:24.921575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1758f686-2c8e-431a-81f3-b64ec965f71d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:24.926783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:24.927375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:24.927648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:24.927915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:24.936822 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:24.938437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:24.946672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1264/2499] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:23:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:23:30.378064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff869c0-67a4-4727-b7be-aad9f9e838a1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:30.379095 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff869c0-67a4-4727-b7be-aad9f9e838a1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:30.379578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff869c0-67a4-4727-b7be-aad9f9e838a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:30.401737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-6ff869c0-67a4-4727-b7be-aad9f9e838a1 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:30.405500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6ff869c0-67a4-4727-b7be-aad9f9e838a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:30.414638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1236/2500] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:23:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:30.436735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:30.451114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a87547-7db8-4737-ac85-f2d6315bceea service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:30.458605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:30.460232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:30.460740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:30.461226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:30.473028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:30.474922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:30.485580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1265/2501] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:23:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:23:40.396582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=63 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:40.484350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=15 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:41.919085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:41.919921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:41.920194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:41.940215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:41.941482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:41.950475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1237/2502] 10.222.0.13 () {58 vars in 967 bytes} [Tue Oct 4 10:23:41 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:41.968294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:41.982587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6c485-5827-40f9-b6af-619e407a58f2 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:41.988036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:41.989253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:41.989618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:41.990073 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:41.999758 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:42.001317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:42.009605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1266/2503] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:23:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:23:43.915337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:43.916908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:43.982756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2576e730-4300-4cb4-a341-1d66e6f2b585 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:43.987290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:43.987905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:43.988181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:43.988454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] PATH_INFO: `/v3/users/4e47148f5c1e47b78a6a038329f0adc9` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:43.989565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e47148f5c1e47b78a6a038329f0adc9)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:43.991520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:44.036886 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e47148f5c1e47b78a6a038329f0adc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.037521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e47148f5c1e47b78a6a038329f0adc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.044879 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e47148f5c1e47b78a6a038329f0adc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.045506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e47148f5c1e47b78a6a038329f0adc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.057194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d8d427de-ae22-41f5-beb7-2f2a9f287f78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e47148f5c1e47b78a6a038329f0adc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.070519 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1238/2504] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:23:43 2022] DELETE /identity/v3/users/4e47148f5c1e47b78a6a038329f0adc9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:44.433377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:44.455699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302ef735-c17d-4a06-98e2-ed25d219437b service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:44.527151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:44.528518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.528885 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.529229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] PATH_INFO: `/v3/projects/acab4db636e64f4183798081133e0b67` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.530481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] RBAC: Authorizing `identity:delete_project(project_id=acab4db636e64f4183798081133e0b67)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:44.532402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:44.582255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acab4db636e64f4183798081133e0b67', 'request_id': 'req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.588545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acab4db636e64f4183798081133e0b67', 'request_id': 'req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.594986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acab4db636e64f4183798081133e0b67', 'request_id': 'req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.609198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] The token cache is being invalidate because project acab4db636e64f4183798081133e0b67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:23:44.609580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.610308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0d1dbaf5-535d-45a1-b8bf-dbbf8f6749da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:23:44.611849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1267/2505] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:23:44 2022] DELETE /identity/v3/projects/acab4db636e64f4183798081133e0b67 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:44.617361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.618177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.618496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.639977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:44.641180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:44.716979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1239/2506] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:44.718151 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.718518 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.718798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.741363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:44.742652 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:44.750855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1268/2507] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:44.761751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:44.773249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb76911d-3afd-49fb-8f0b-79041a17bd6d None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:44.779000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:44.779614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.779884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.780161 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.780804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:23:44.781104 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:44.782854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:44.793484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:44.797674 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1240/2508] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:23:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:23:44.802473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:44.815287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e07671-5de5-4858-95c1-b9f5721a057a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:44.821930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:44.822516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.822886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.823139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.823781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:44.826453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:44.858005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1269/2509] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:23:44 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:23:44.859133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:44.868853 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1af964-4231-4b8d-ad76-21c40892d22d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:44.930671 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:44.931293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.931540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.931810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.932581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:44.935363 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:44.965783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1241/2510] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:23:44 2022] POST /identity/v3/users => generated 284 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:23:44.970251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:44.979610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51212d1-7113-4aba-9291-ae8d5b20a2f2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:44.984864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:44.985583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:44.985977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:44.986384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:44.987123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:44.988981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:45.001527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1270/2511] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:23:45.005251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:45.018068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58a9ad2-2488-47a4-a13b-9f41762bbb66 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:45.022764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285bcd08-27be-4639-9063-ff8bfac1debb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:45.023375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bcd08-27be-4639-9063-ff8bfac1debb admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:45.023646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bcd08-27be-4639-9063-ff8bfac1debb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:45.023918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285bcd08-27be-4639-9063-ff8bfac1debb admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:45.024584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bcd08-27be-4639-9063-ff8bfac1debb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:45.026160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285bcd08-27be-4639-9063-ff8bfac1debb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:45.039559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1242/2512] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:23:45.043698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:45.053747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658f682-c15e-40fe-a474-ea7516028d72 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:45.059844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:45.060573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:45.060851 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:45.061121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] PATH_INFO: `/v3/projects/f4ecdf9a1b3544f1a8966e20c7e864ee/users/65515c46fca94913857becb39c018e63/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:45.076640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ecdf9a1b3544f1a8966e20c7e864ee, user_id=65515c46fca94913857becb39c018e63, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:45.079321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:45.095992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1271/2513] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:23:45 2022] PUT /identity/v3/projects/f4ecdf9a1b3544f1a8966e20c7e864ee/users/65515c46fca94913857becb39c018e63/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:23:45.107252 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:45.107878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:45.108319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:45.139462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] MFA Rules not processed for user `65515c46fca94913857becb39c018e63`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:23:45.140694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:45.207531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1243/2514] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:23:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:23:45.221369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:45.222992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:45.240170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:45.292772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619c9e0a-d4c6-4318-899c-a08bbd0776fc admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:45.297324 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd27bb8-5bbc-4fad-9411-17f5de67e5e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:45.297931 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd27bb8-5bbc-4fad-9411-17f5de67e5e0 service glance] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:45.298224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd27bb8-5bbc-4fad-9411-17f5de67e5e0 service glance] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:45.298506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd27bb8-5bbc-4fad-9411-17f5de67e5e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:45.306966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd27bb8-5bbc-4fad-9411-17f5de67e5e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:45.308378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd27bb8-5bbc-4fad-9411-17f5de67e5e0 service glance] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:45.317038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1272/2515] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:23:48.082862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:23:48.085028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:48.164657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a1194c-babd-4c69-a443-f9e8dd2cd6c6 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:23:48.170254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:23:48.170806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:23:48.171091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:23:48.171352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:23:48.173099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:23:48.184772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:23:48.186427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:23:48.190745 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:48.200407 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1244/2516] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:23:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:23:55.313190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=20 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:23:58.198996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:24:00.868741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:00.869238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:00.869342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:00.889144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] MFA Rules not processed for user `a7a1fdb5bd22468c99a0024d24f6d606`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:24:00.890256 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:00.945706 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1273/2517] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:24:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:24:03.857800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:24:03.859338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:03.922829 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6120aa-9f51-4515-b7f5-7eb55d35920b service placement] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:24:03.926614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:24:03.927221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:03.927498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:03.927768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:03.929520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:03.995795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:24:03.997737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b541739-283e-4619-bf7b-fbd30b8e9a3f service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:24:04.007501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1245/2518] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:24:03 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:24:10.938978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=25 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:24:14.003064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=50 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:24:22.123854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:24:22.184111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae2a09-392a-4a6e-9bd7-eeaa317274b2 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:24:22.189125 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:24:22.189820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:22.190100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:22.190408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] PATH_INFO: `/v3/users/65515c46fca94913857becb39c018e63` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:22.191539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65515c46fca94913857becb39c018e63)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:24:22.193465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:24:22.246623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65515c46fca94913857becb39c018e63'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.247329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65515c46fca94913857becb39c018e63'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.253994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65515c46fca94913857becb39c018e63'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.254634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65515c46fca94913857becb39c018e63'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.267299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65515c46fca94913857becb39c018e63'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.281416 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1274/2519] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:24:22 2022] DELETE /identity/v3/users/65515c46fca94913857becb39c018e63 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:24:22.286317 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:22.286857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:22.287040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:22.306340 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:24:22.307539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:22.368679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1246/2520] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:24:22.377608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:24:22.379137 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:22.437574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2a12-1b77-4325-b965-52c94a8285a5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:24:22.441552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05db6f7-a044-44fd-9027-c28de708d9f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:24:22.442166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db6f7-a044-44fd-9027-c28de708d9f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:22.442471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db6f7-a044-44fd-9027-c28de708d9f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:22.442739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05db6f7-a044-44fd-9027-c28de708d9f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:22.452896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05db6f7-a044-44fd-9027-c28de708d9f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:24:22.454396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05db6f7-a044-44fd-9027-c28de708d9f9 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:24:22.464085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1275/2521] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:24:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:24:22.730404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:24:22.731943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:22.744071 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb880-a5ca-42d4-985c-9f9695a1a268 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:24:22.749306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:24:22.749937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:22.750234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:22.750511 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] PATH_INFO: `/v3/projects/f4ecdf9a1b3544f1a8966e20c7e864ee` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:22.751648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ecdf9a1b3544f1a8966e20c7e864ee)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:24:22.753816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:24:22.802124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ecdf9a1b3544f1a8966e20c7e864ee', 'request_id': 'req-72706fa7-7fd3-42d3-af82-6fc1b45c3697'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.807616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ecdf9a1b3544f1a8966e20c7e864ee', 'request_id': 'req-72706fa7-7fd3-42d3-af82-6fc1b45c3697'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.815635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ecdf9a1b3544f1a8966e20c7e864ee', 'request_id': 'req-72706fa7-7fd3-42d3-af82-6fc1b45c3697'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.826635 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] The token cache is being invalidate because project f4ecdf9a1b3544f1a8966e20c7e864ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:24:22.826930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.827560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-72706fa7-7fd3-42d3-af82-6fc1b45c3697 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:24:22.829043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1247/2522] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:24:22 2022] DELETE /identity/v3/projects/f4ecdf9a1b3544f1a8966e20c7e864ee => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:24:32.459847 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:24:32.823667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:24:39.029267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e24f2-d4b4-489c-b4b2-d601f5106b3c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:24:39.029894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e24f2-d4b4-489c-b4b2-d601f5106b3c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:24:39.029894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e24f2-d4b4-489c-b4b2-d601f5106b3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:24:39.048565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ec5e24f2-d4b4-489c-b4b2-d601f5106b3c None None] MFA Rules not processed for user `28c104b3e1284882b72d6b6933475029`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:24:39.049745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ec5e24f2-d4b4-489c-b4b2-d601f5106b3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:24:39.104421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1276/2523] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:24:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:24:49.097068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=25 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:11.861622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d5d7-3974-4c2a-940a-222307d3dca8 None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:11.862248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d5d7-3974-4c2a-940a-222307d3dca8 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:11.862248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf9d5d7-3974-4c2a-940a-222307d3dca8 None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:11.863531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1248/2524] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 04 10:25:12.114674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.115239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.115239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.135681 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] MFA Rules not processed for user `c873c7e9014b46c8b1bf717f1e0fe514`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:12.136929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:12.148243 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9d5d7-3974-4c2a-940a-222307d3dca8 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.150223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9cf9d5d7-3974-4c2a-940a-222307d3dca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:12.215684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1277/2525] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:25:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:12.217117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf9d5d7-3974-4c2a-940a-222307d3dca8 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.223075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.223216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.223482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.223752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] PATH_INFO: `/v3/users/a7a1fdb5bd22468c99a0024d24f6d606` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.224893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7a1fdb5bd22468c99a0024d24f6d606)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.226831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:12.228001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.229361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:12.242403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887208ad-19c2-45ba-b0fd-febb89e7a584 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.247565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.248189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.248464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.248739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.260788 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.262500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:12.274840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1278/2526] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:12 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:25:12.276691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7a1fdb5bd22468c99a0024d24f6d606'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.277352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7a1fdb5bd22468c99a0024d24f6d606'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.284893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7a1fdb5bd22468c99a0024d24f6d606'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.285560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7a1fdb5bd22468c99a0024d24f6d606'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.300160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-5c0577ec-0d9c-4a8a-9a32-ada2492a1dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7a1fdb5bd22468c99a0024d24f6d606'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.313653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1249/2527] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:25:12 2022] DELETE /identity/v3/users/a7a1fdb5bd22468c99a0024d24f6d606 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:25:12.484993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.496805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5ca60-3854-4786-b6ef-4e37dc6a0dd2 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.561916 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.562583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.562863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.563150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] PATH_INFO: `/v3/projects/9e814fc8005d4941b5eec7aa06144683` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.564248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e814fc8005d4941b5eec7aa06144683)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.567306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:12.617025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e814fc8005d4941b5eec7aa06144683', 'request_id': 'req-5d235330-c3bf-418d-a3d6-d519e350f106'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.623307 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e814fc8005d4941b5eec7aa06144683', 'request_id': 'req-5d235330-c3bf-418d-a3d6-d519e350f106'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.630367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e814fc8005d4941b5eec7aa06144683', 'request_id': 'req-5d235330-c3bf-418d-a3d6-d519e350f106'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.643881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] The token cache is being invalidate because project 9e814fc8005d4941b5eec7aa06144683 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:25:12.644174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.644827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5d235330-c3bf-418d-a3d6-d519e350f106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:12.646341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1279/2528] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:25:12 2022] DELETE /identity/v3/projects/9e814fc8005d4941b5eec7aa06144683 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:25:12.651426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.651708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.651978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.671215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:12.672338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:12.731789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1250/2529] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:12.740570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.740848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.741124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.761096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:12.762350 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:12.770299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1280/2530] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:12.774656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.784412 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fa4fe3-4564-491a-a4ad-c7beec176b99 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.789012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.789598 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.789875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.790147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.790959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:25:12.791260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.792878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:12.804774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1251/2531] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:25:12.809245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.818997 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472ece5-2325-415e-aa47-1aa4a433695d None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.823879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.824463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.824734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.825001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.825624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.827547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:12.831948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:12.857124 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1281/2532] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:25:12 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:25:12.859651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.870280 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190a979-5c95-43bb-aeba-f8d597419c96 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.873864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:12.939727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.940930 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.941208 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.941502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.942137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.944065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:12.966400 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1252/2533] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:25:12 2022] POST /identity/v3/users => generated 299 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:25:12.969826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:12.982253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6840414-a3eb-4c18-b398-a72f8830870b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:12.986966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:12.987560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:12.987832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:12.988099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:12.988733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:12.990351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:13.000233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1282/2534] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:25:13.004834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:13.015039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0d329a-762f-46fa-a3c0-14ca234f9654 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:13.020714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5718ea-c3f7-447c-8c22-14c8de518bad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:13.021501 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5718ea-c3f7-447c-8c22-14c8de518bad admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:13.021779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5718ea-c3f7-447c-8c22-14c8de518bad admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:13.022050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5718ea-c3f7-447c-8c22-14c8de518bad admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:13.022747 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5718ea-c3f7-447c-8c22-14c8de518bad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:13.024327 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5718ea-c3f7-447c-8c22-14c8de518bad admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:13.036268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1253/2535] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:25:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:25:13.040461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:13.051516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0605e40c-02e8-48c8-9013-b6c9fd0afd42 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:13.056775 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:13.057410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:13.057687 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:13.057953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] PATH_INFO: `/v3/projects/189932bdc28e4eee8c1d4ab92043165a/users/23e9a2655e5547538d4759dea9e89fc9/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:13.072683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] RBAC: Authorizing `identity:create_grant(project_id=189932bdc28e4eee8c1d4ab92043165a, user_id=23e9a2655e5547538d4759dea9e89fc9, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:13.075363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:13.088318 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1283/2536] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:25:13 2022] PUT /identity/v3/projects/189932bdc28e4eee8c1d4ab92043165a/users/23e9a2655e5547538d4759dea9e89fc9/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:25:13.099529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:13.099812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:13.100087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:13.127812 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] MFA Rules not processed for user `23e9a2655e5547538d4759dea9e89fc9`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:13.128964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:13.194416 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1254/2537] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:13.206130 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:13.208077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:13.267386 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5ac3c8-2582-4ef8-a55d-12905527944d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:13.271662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38919b5e-d06b-43d5-9d7d-7eedc75fa2cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:13.272266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38919b5e-d06b-43d5-9d7d-7eedc75fa2cd service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:13.272540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38919b5e-d06b-43d5-9d7d-7eedc75fa2cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:13.272809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38919b5e-d06b-43d5-9d7d-7eedc75fa2cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:13.281204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38919b5e-d06b-43d5-9d7d-7eedc75fa2cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:13.282715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38919b5e-d06b-43d5-9d7d-7eedc75fa2cd service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:13.293451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1284/2538] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:13 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:20.357084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:20.436167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f984e26b-701f-40c6-9f37-b759984a9646 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:20.440849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:20.442617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:20.442902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:20.443178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] PATH_INFO: `/v3/users/23e9a2655e5547538d4759dea9e89fc9` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:20.444350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23e9a2655e5547538d4759dea9e89fc9)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:20.446696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:20.464748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:20.502084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23e9a2655e5547538d4759dea9e89fc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:20.503215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23e9a2655e5547538d4759dea9e89fc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:20.511246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23e9a2655e5547538d4759dea9e89fc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:20.512046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23e9a2655e5547538d4759dea9e89fc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:20.525806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23e9a2655e5547538d4759dea9e89fc9'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:20.545370 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1255/2539] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:25:20 2022] DELETE /identity/v3/users/23e9a2655e5547538d4759dea9e89fc9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:25:20.549853 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:20.551002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:20.551203 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:20.573989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:20.575361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:20.638623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1285/2540] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:20.648485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:20.650009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:20.711627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e74a58d-2287-4738-9ea7-6dfe13bc2d42 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:20.715294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf30ab3-b106-4fb4-9eee-5629f7512ce9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:20.715898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30ab3-b106-4fb4-9eee-5629f7512ce9 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:20.716176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30ab3-b106-4fb4-9eee-5629f7512ce9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:20.716452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf30ab3-b106-4fb4-9eee-5629f7512ce9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:20.726647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf30ab3-b106-4fb4-9eee-5629f7512ce9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:20.728123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf30ab3-b106-4fb4-9eee-5629f7512ce9 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:20.737505 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1256/2541] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:21.018089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:21.020317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:21.038624 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63b57d-f1a4-4cd7-b64a-634e04a47c5f None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:21.050305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:21.050948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:21.051220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:21.051487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] PATH_INFO: `/v3/projects/189932bdc28e4eee8c1d4ab92043165a` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:21.052602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=189932bdc28e4eee8c1d4ab92043165a)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:21.054564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:21.110861 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:21.112799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '189932bdc28e4eee8c1d4ab92043165a', 'request_id': 'req-63bd63f2-2bb3-4887-8725-12933cd0d6a1'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:21.120367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '189932bdc28e4eee8c1d4ab92043165a', 'request_id': 'req-63bd63f2-2bb3-4887-8725-12933cd0d6a1'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:21.129035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '189932bdc28e4eee8c1d4ab92043165a', 'request_id': 'req-63bd63f2-2bb3-4887-8725-12933cd0d6a1'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:21.143494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] The token cache is being invalidate because project 189932bdc28e4eee8c1d4ab92043165a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:25:21.143692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:21.145219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:21.146769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1286/2542] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:25:21 2022] DELETE /identity/v3/projects/189932bdc28e4eee8c1d4ab92043165a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:25:23.783865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:23.784391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:23.784497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:23.804998 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] MFA Rules not processed for user `ef09a3d1aed344eeb6a5100ddd420e16`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:23.806510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:23.862377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1257/2543] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:31.139849 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=7,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:33.855692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=9,keystone:SELECT=71,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:36.319264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:36.320702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:36.381363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd63f2-2bb3-4887-8725-12933cd0d6a1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:36.385594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:36.386223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:36.386516 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:36.386787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:36.388716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:36.467663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:36.469348 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:36.480328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1287/2544] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:37.649447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:37.659436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7feccf3a-2158-4d67-87ee-b165b737727a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:37.663202 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:37.663794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:37.664068 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:37.664338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:37.666035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:37.680012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:37.681424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:37.693246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1258/2545] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:41.587514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:41.589063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:41.605208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42afaf28-2ea2-408c-8ddc-486d8a73719b service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:41.613591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:41.614242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:41.614528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:41.614858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:41.620107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:41.642420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:41.644797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:41.655844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1288/2546] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:44.134578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:44.136304 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:44.149801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bd7b7d-a055-4a31-aa44-04f7d0f2f6a3 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:44.155254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:44.155846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:44.156117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:44.156382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:44.158048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:44.216069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:44.217648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab131b99-4202-4955-8ce4-0d22b8e494e9 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:44.229037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1259/2547] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:25:49.217189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:49.227242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3479d-fed9-421c-a405-6cb8485e015e service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:49.230933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:49.231600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:49.231941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:49.232279 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:49.234356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:49.300288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:49.301815 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:49.311829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1289/2548] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:53.196333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.196885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.197359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:53.240698 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:53.243570 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:53.245063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.246522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.322257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:53.330039 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1260/2549] 10.222.0.13 () {58 vars in 967 bytes} [Tue Oct 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:53.384794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:53.386894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.406039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6d7b11-2466-4823-a90c-44414ba11586 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:53.412105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:53.412726 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.413132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.413467 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:53.415480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.417169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfec6578-f4dd-499a-8d2f-b36c036a4112 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:53.422348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:53.423053 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.423455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.423793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] PATH_INFO: `/v3/users/28c104b3e1284882b72d6b6933475029` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:53.425124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] RBAC: Authorizing `identity:delete_user(user_id=28c104b3e1284882b72d6b6933475029)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:53.429881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:53.430019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:53.431538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:53.444268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1290/2550] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:53.445101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:53.446677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.488684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28c104b3e1284882b72d6b6933475029'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.489357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28c104b3e1284882b72d6b6933475029'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.497321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28c104b3e1284882b72d6b6933475029'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.498066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28c104b3e1284882b72d6b6933475029'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.511250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28c104b3e1284882b72d6b6933475029'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.524440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e84b3-0d7d-4535-93a7-d751736cdb1e service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:53.525982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1261/2551] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:25:53 2022] DELETE /identity/v3/users/28c104b3e1284882b72d6b6933475029 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:25:53.587712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:53.588356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.588637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.588905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:53.590784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.663984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:53.666373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:53.668057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f94408-abab-478f-a4a7-9f5161614ea8 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:53.677285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1291/2552] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:25:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:25:53.704959 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:53.707969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.713424 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:53.783145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d192e648-9acd-4b8e-b68e-6ad8b94d396e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:53.789388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:53.790023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.790350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.790630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] PATH_INFO: `/v3/projects/d85cf718f1b248c192c138bea872fc0d` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:53.791901 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85cf718f1b248c192c138bea872fc0d)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:53.793798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:53.850156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85cf718f1b248c192c138bea872fc0d', 'request_id': 'req-792f0ace-3d79-4a59-8b8f-c0cb93909f86'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.856003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85cf718f1b248c192c138bea872fc0d', 'request_id': 'req-792f0ace-3d79-4a59-8b8f-c0cb93909f86'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.864982 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85cf718f1b248c192c138bea872fc0d', 'request_id': 'req-792f0ace-3d79-4a59-8b8f-c0cb93909f86'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.876926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] The token cache is being invalidate because project d85cf718f1b248c192c138bea872fc0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:25:53.877338 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.878733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-792f0ace-3d79-4a59-8b8f-c0cb93909f86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:25:53.880195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1262/2553] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:25:53 2022] DELETE /identity/v3/projects/d85cf718f1b248c192c138bea872fc0d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:25:53.885374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198631-bab7-4233-bcca-b21683a1534e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.885648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198631-bab7-4233-bcca-b21683a1534e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.885917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97198631-bab7-4233-bcca-b21683a1534e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:53.907737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-97198631-bab7-4233-bcca-b21683a1534e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:53.909051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-97198631-bab7-4233-bcca-b21683a1534e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:53.971371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1292/2554] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:25:53.979224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:53.979507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:53.979781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.000918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:54.002121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:54.011284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1263/2555] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:54.015841 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198631-bab7-4233-bcca-b21683a1534e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.025914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97198631-bab7-4233-bcca-b21683a1534e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.030718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.031412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.031684 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.031949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.032589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:25:54.032887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.034566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.046368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1293/2556] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:25:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:25:54.051299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.060465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efcb97c-317f-49fc-9120-986a9cc64e2b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.065019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.066254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.066534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.066801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.067536 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.069418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.098613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1264/2557] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:25:54 2022] POST /identity/v3/projects => generated 389 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:25:54.102737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.112852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cf6cf-9a01-410f-9ed2-9d73a2ec258b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.167584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.168188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.168463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.168741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.169355 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.171311 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.204978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1294/2558] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:25:54 2022] POST /identity/v3/users => generated 295 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:25:54.207200 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.217652 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31636b1b-b00e-4511-90fc-a83b17f88279 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.222750 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.223345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.223626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.223900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.224537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.226139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.236492 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1265/2559] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:25:54.240500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.250837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931754d2-8b60-4343-8aa8-91f81d1b1c87 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.256634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8077ecfa-6c78-4dd9-b714-3dba16483cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.257251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ecfa-6c78-4dd9-b714-3dba16483cc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.257534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ecfa-6c78-4dd9-b714-3dba16483cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.257810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8077ecfa-6c78-4dd9-b714-3dba16483cc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.258499 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077ecfa-6c78-4dd9-b714-3dba16483cc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.260115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8077ecfa-6c78-4dd9-b714-3dba16483cc6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.272446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1295/2560] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:25:54.277164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.287271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81405896-4cb2-4fc0-b6f6-1419378c59fe admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.292130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.292779 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.293241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.293530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] PATH_INFO: `/v3/projects/25a4307806824ec4bd3683b8ca7b9752/users/c2fdbf946666403bad994a897335672c/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.310659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] RBAC: Authorizing `identity:create_grant(project_id=25a4307806824ec4bd3683b8ca7b9752, user_id=c2fdbf946666403bad994a897335672c, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.313288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.328002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1266/2561] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:25:54 2022] PUT /identity/v3/projects/25a4307806824ec4bd3683b8ca7b9752/users/c2fdbf946666403bad994a897335672c/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:25:54.334524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32c157-22f3-41b2-b86d-2676bfe5ed98 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.334753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32c157-22f3-41b2-b86d-2676bfe5ed98 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.335080 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32c157-22f3-41b2-b86d-2676bfe5ed98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.354611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-cb32c157-22f3-41b2-b86d-2676bfe5ed98 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:54.355836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-cb32c157-22f3-41b2-b86d-2676bfe5ed98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:54.417698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1296/2562] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:25:54.427772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:25:54.429549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:25:54.463203 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:54.490874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07237735-c000-47ca-b553-4b4cc224458b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:25:54.494948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:25:54.495666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:54.495949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:54.496222 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:54.506415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:25:54.507840 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:25:54.518000 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1267/2563] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:25:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:25:59.950429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38135030-420d-4d0e-8724-3328bf579e8c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:25:59.951409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38135030-420d-4d0e-8724-3328bf579e8c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:25:59.951898 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38135030-420d-4d0e-8724-3328bf579e8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:25:59.962167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:25:59.993448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-38135030-420d-4d0e-8724-3328bf579e8c None None] MFA Rules not processed for user `c2fdbf946666403bad994a897335672c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:25:59.995109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-38135030-420d-4d0e-8724-3328bf579e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:00.098509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1297/2564] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:00.106007 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:00.108888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:00.197659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3d300c-f5c6-4a97-981e-28d2d02d23a3 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:00.204380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:00.205087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:00.205421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:00.206585 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:00.216046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:00.217757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:00.234099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1268/2565] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:01.783447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:01.784625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:01.784625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:01.808160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] MFA Rules not processed for user `c2fdbf946666403bad994a897335672c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:01.809761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:01.822069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1298/2566] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:02.334771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:02.337025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:02.380100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256ad29f-2d68-4291-b68a-26849158c63d service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:02.391712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:02.392938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:02.394895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:02.397328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:02.397804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:02.398270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:02.400337 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:02.413971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:02.415352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:02.419139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d8a69d-4853-4fca-85fd-3145bfa8ca2b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:02.433284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:02.434277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1269/2567] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:26:02.434852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:02.435403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:02.435669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] PATH_INFO: `/v3/users/7ed8b85b47694218add98b6ed986eff3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:02.436924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ed8b85b47694218add98b6ed986eff3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:02.439193 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:02.520982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ed8b85b47694218add98b6ed986eff3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:02.521671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ed8b85b47694218add98b6ed986eff3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:02.530011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ed8b85b47694218add98b6ed986eff3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:02.530754 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ed8b85b47694218add98b6ed986eff3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:02.547699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ed8b85b47694218add98b6ed986eff3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:02.569126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1299/2568] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:26:02 2022] DELETE /identity/v3/users/7ed8b85b47694218add98b6ed986eff3 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:26:02.798748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:02.820499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7a052-4756-471f-9383-4186a020346c service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:02.876257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:02.878012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:02.902696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:02.903414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:02.903915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:02.904244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] PATH_INFO: `/v3/projects/c3a2d556879e4a57ae3e889536468a8c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:02.905441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a2d556879e4a57ae3e889536468a8c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:02.907682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:02.967695 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:02.983628 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:02.985313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a2d556879e4a57ae3e889536468a8c', 'request_id': 'req-0c0e2bba-a48e-45d0-9162-be9009a193d0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:02.993349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa7bd0f-cf92-4d5f-8239-08e479b15c22 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:02.993561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a2d556879e4a57ae3e889536468a8c', 'request_id': 'req-0c0e2bba-a48e-45d0-9162-be9009a193d0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.007593 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a2d556879e4a57ae3e889536468a8c', 'request_id': 'req-0c0e2bba-a48e-45d0-9162-be9009a193d0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.041619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] The token cache is being invalidate because project c3a2d556879e4a57ae3e889536468a8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:26:03.044587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.046298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.048785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1270/2569] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:26:02 2022] DELETE /identity/v3/projects/c3a2d556879e4a57ae3e889536468a8c => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:03.055373 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:03.057056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:03.106150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:03.107109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:03.107522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:03.107929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:03.114240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:03.129424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:03.131296 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:03.148033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1300/2570] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:26:03.153258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e2bba-a48e-45d0-9162-be9009a193d0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:03.164331 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:03.164970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:03.165298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:03.165573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] PATH_INFO: `/v3/users/fbcaaf3b79ce4ca59d74081eeb6a5091` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:03.167114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbcaaf3b79ce4ca59d74081eeb6a5091)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:03.169545 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:03.250438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbcaaf3b79ce4ca59d74081eeb6a5091'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.251290 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbcaaf3b79ce4ca59d74081eeb6a5091'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.259607 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbcaaf3b79ce4ca59d74081eeb6a5091'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.260353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbcaaf3b79ce4ca59d74081eeb6a5091'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.280214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbcaaf3b79ce4ca59d74081eeb6a5091'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.312321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1271/2571] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:26:03 2022] DELETE /identity/v3/users/fbcaaf3b79ce4ca59d74081eeb6a5091 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:03.572477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:03.586809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29829345-7cbd-43e3-ae36-5589cb9d3770 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:03.688108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:03.688848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:03.689195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:03.689581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] PATH_INFO: `/v3/projects/7970519189e04aa7afc0e66511ae49d0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:03.691558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] RBAC: Authorizing `identity:delete_project(project_id=7970519189e04aa7afc0e66511ae49d0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:03.693753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:03.751140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7970519189e04aa7afc0e66511ae49d0', 'request_id': 'req-b6011ad4-73f3-42e1-85a9-e1af8d786cab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.761960 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7970519189e04aa7afc0e66511ae49d0', 'request_id': 'req-b6011ad4-73f3-42e1-85a9-e1af8d786cab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.773464 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7970519189e04aa7afc0e66511ae49d0', 'request_id': 'req-b6011ad4-73f3-42e1-85a9-e1af8d786cab'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.791168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] The token cache is being invalidate because project 7970519189e04aa7afc0e66511ae49d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:26:03.791831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.792724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:03.796159 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1301/2572] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:26:03 2022] DELETE /identity/v3/projects/7970519189e04aa7afc0e66511ae49d0 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:05.461358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:05.462948 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:05.526241 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b464b53-bba2-4b5a-9dec-eaa989bf0824 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:05.531834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:05.587019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:05.587626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:05.587898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:05.588307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:05.590070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:05.601175 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:05.602651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:05.613326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1272/2573] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:06.239371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:06.240992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:06.271300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:06.329873 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6011ad4-73f3-42e1-85a9-e1af8d786cab admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:06.336606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:06.337227 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:06.337503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:06.337776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:06.339565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:06.353022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:06.354649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:06.365977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1302/2574] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:11.506757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:11.523645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502dd1ee-97dd-4c6b-8eb8-c7b53d228975 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:11.529476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:11.530086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:11.530397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:11.530669 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:11.532428 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:11.545612 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:11.547217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:11.562049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1273/2575] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:16.360972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=25 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:21.554894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=28 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:52.775727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:52.777155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:52.789611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d8b3c3-ff3a-4220-b7c7-f5a152ea95fe service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:52.793189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:52.793774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:52.794050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:52.794366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:52.796027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:52.860492 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:52.862024 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:52.871640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1303/2576] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:56.045869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.047949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:56.061878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40749499-906e-4d3d-a3b3-b33922bc9fda service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.067244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.067888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.068163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.068442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] PATH_INFO: `/v3/users/ef09a3d1aed344eeb6a5100ddd420e16` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.069524 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef09a3d1aed344eeb6a5100ddd420e16)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.071530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.123446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef09a3d1aed344eeb6a5100ddd420e16'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.124092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef09a3d1aed344eeb6a5100ddd420e16'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.131507 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef09a3d1aed344eeb6a5100ddd420e16'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.132198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef09a3d1aed344eeb6a5100ddd420e16'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.147643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-4a145858-ba71-4aa4-a814-ddfe9f69e533 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef09a3d1aed344eeb6a5100ddd420e16'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.161637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1274/2577] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:26:56 2022] DELETE /identity/v3/users/ef09a3d1aed344eeb6a5100ddd420e16 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:56.334086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.345114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1f5e54-4017-472d-89d7-7e879e5a23ad service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.401941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.402534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.402812 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.403081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] PATH_INFO: `/v3/projects/64ddf0d073bd420c90de975012dc2b9e` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.404186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=64ddf0d073bd420c90de975012dc2b9e)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.406081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.461062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64ddf0d073bd420c90de975012dc2b9e', 'request_id': 'req-7a094e84-35c6-4baa-834d-ccceec392bbd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.467108 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64ddf0d073bd420c90de975012dc2b9e', 'request_id': 'req-7a094e84-35c6-4baa-834d-ccceec392bbd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.472886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64ddf0d073bd420c90de975012dc2b9e', 'request_id': 'req-7a094e84-35c6-4baa-834d-ccceec392bbd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.485802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] The token cache is being invalidate because project 64ddf0d073bd420c90de975012dc2b9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:26:56.486226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.486938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7a094e84-35c6-4baa-834d-ccceec392bbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:56.488438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1304/2578] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:26:56 2022] DELETE /identity/v3/projects/64ddf0d073bd420c90de975012dc2b9e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:56.493310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.493602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.493876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.513618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:56.514854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:56.571684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1275/2579] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:26:56.580017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.580298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.580567 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.601255 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:56.602489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:56.610562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1305/2580] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:56.614737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.624512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8ca3-f12e-415a-8025-a4b31902a6ac None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.630029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.630663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.630945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.631214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.631849 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:26:56.632146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.633731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.646021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1276/2581] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:26:56.650375 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.659769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41dc7b2-d1b1-4cc1-b257-a10b82aeae1c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.664364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.664948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.665225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.665495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.666140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.668065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.682678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:56.697041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1306/2582] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:26:56 2022] POST /identity/v3/projects => generated 381 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:26:56.701276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.711457 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931b066-b9b7-4491-af4d-2f5f1dcfe72d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.768137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.768733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.769010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.769277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.770019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.771969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.779774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:56.794531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1277/2583] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:26:56 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:26:56.798669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.808804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0594d0-e47a-4d2b-aa29-244acf63c73f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.813336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.813923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.814219 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.814500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.815164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.816787 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.826948 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1307/2584] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:26:56.831278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.840876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16dc7763-d42f-48cf-bb4f-17978b69931d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.846633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3f699d-6d8e-49df-8e6a-5c614b58341b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.847229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f699d-6d8e-49df-8e6a-5c614b58341b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.847559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f699d-6d8e-49df-8e6a-5c614b58341b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.847851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3f699d-6d8e-49df-8e6a-5c614b58341b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.848517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f699d-6d8e-49df-8e6a-5c614b58341b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.850121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3f699d-6d8e-49df-8e6a-5c614b58341b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.861127 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1278/2585] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:26:56.865341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:56.875658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0c2f-1494-441e-8c44-c0df5e2f0420 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:56.880009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:56.880640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.880915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.881179 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] PATH_INFO: `/v3/projects/59a7d0696ca94dbeafb8c8b4022bcbd4/users/da2fbd030a3340fb8d548c209a7e375d/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.896002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a7d0696ca94dbeafb8c8b4022bcbd4, user_id=da2fbd030a3340fb8d548c209a7e375d, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:56.898690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:56.913009 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1308/2586] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:26:56 2022] PUT /identity/v3/projects/59a7d0696ca94dbeafb8c8b4022bcbd4/users/da2fbd030a3340fb8d548c209a7e375d/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:56.924311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb532db-099e-42be-823c-87c449e79683 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:56.924626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb532db-099e-42be-823c-87c449e79683 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:56.924904 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb532db-099e-42be-823c-87c449e79683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:56.953653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-dbb532db-099e-42be-823c-87c449e79683 None None] MFA Rules not processed for user `da2fbd030a3340fb8d548c209a7e375d`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:56.955877 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dbb532db-099e-42be-823c-87c449e79683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:57.025910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1279/2587] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:26:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:57.029344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.096006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65450cc5-cbd4-4165-aee0-3ac775496d51 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.100782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.101395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.101753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.102029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.102685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.104647 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.131581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1309/2588] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:26:57 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:26:57.135689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb532db-099e-42be-823c-87c449e79683 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.147014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb532db-099e-42be-823c-87c449e79683 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.203258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.204335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.204611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.204881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.205513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.207435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.231147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1280/2589] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:26:57 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:26:57.235457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.246451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3776cb6f-83cd-4aec-a0e0-edf47f27b8bf admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.251946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.252486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.252764 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.253031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.253668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.255300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.267921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1310/2590] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:26:57.272176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.282916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49cccef-d35f-4739-8d2a-1b3175ccfb8f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.287523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.288169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.288448 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.288728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] PATH_INFO: `/v3/projects/744bd3ed52e04da4a0c99fb90a09b94f/users/525b3a3aaf6f4970ba1208d215be098f/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.304683 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744bd3ed52e04da4a0c99fb90a09b94f, user_id=525b3a3aaf6f4970ba1208d215be098f, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.307404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.326575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1281/2591] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:26:57 2022] PUT /identity/v3/projects/744bd3ed52e04da4a0c99fb90a09b94f/users/525b3a3aaf6f4970ba1208d215be098f/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:57.331366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.345584 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eb4884-dc4f-4900-b15b-611c8caeb9e9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.408186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57730530-3fb5-433c-a414-692da2630626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.408793 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57730530-3fb5-433c-a414-692da2630626 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.409062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57730530-3fb5-433c-a414-692da2630626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.409328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57730530-3fb5-433c-a414-692da2630626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.409954 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57730530-3fb5-433c-a414-692da2630626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.411579 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57730530-3fb5-433c-a414-692da2630626 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.421214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1311/2592] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:26:57.425466 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.435437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08998c-a327-4d74-ba14-2e3e273e2676 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.440809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.441437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.441709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.441980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] PATH_INFO: `/v3/projects/744bd3ed52e04da4a0c99fb90a09b94f/users/525b3a3aaf6f4970ba1208d215be098f/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.443878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] RBAC: Authorizing `identity:create_grant(project_id=744bd3ed52e04da4a0c99fb90a09b94f, user_id=525b3a3aaf6f4970ba1208d215be098f, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.446556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.457025 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1282/2593] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:26:57 2022] PUT /identity/v3/projects/744bd3ed52e04da4a0c99fb90a09b94f/users/525b3a3aaf6f4970ba1208d215be098f/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:57.467017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.467305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.467582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.495715 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] MFA Rules not processed for user `525b3a3aaf6f4970ba1208d215be098f`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:57.496918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:57.515030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:57.563881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1312/2594] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:57.576028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.577711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:57.599994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:57.638355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eb0f3f-a993-4c1e-bb7c-642b2d9e9989 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.642256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.642869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.643150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.643422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.655426 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.657453 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:57.669139 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1283/2595] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:57.975127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:57.984180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c640d86-948b-4333-89d1-0f1949281d39 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:57.988810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '525b3a3aaf6f4970ba1208d215be098f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '744bd3ed52e04da4a0c99fb90a09b94f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:57.989498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:57.989677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:57.989982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:57.990661 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:26:57.991001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:57.992753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:58.004747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1313/2596] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:26:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:26:58.008789 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:58.018621 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b417cf0-8569-45a1-a4a0-848f34b04ab7 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:58.023134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '525b3a3aaf6f4970ba1208d215be098f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '744bd3ed52e04da4a0c99fb90a09b94f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:58.023710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:58.023985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:58.024255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:58.024926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:58.026819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:58.052330 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1284/2597] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:26:58 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:26:58.056242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:58.066651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061b9442-b5f9-4b54-b98e-1596f6adb5e7 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:58.120171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '525b3a3aaf6f4970ba1208d215be098f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '744bd3ed52e04da4a0c99fb90a09b94f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:58.120762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:58.121042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:58.121314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:58.121976 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:26:58.122305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:58.123878 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:58.135763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1314/2598] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:26:58.172969 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:58.183295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4cfd9-93e1-43a8-bcba-c6317fa17c12 tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:58.188447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '525b3a3aaf6f4970ba1208d215be098f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '744bd3ed52e04da4a0c99fb90a09b94f', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:58.189171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:58.189454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:58.189727 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] PATH_INFO: `/v3/projects/5e3c1e94bd584bb78d9cf38ca66de1b9` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:58.190864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5e3c1e94bd584bb78d9cf38ca66de1b9)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:58.192738 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:58.244481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e3c1e94bd584bb78d9cf38ca66de1b9', 'request_id': 'req-d52ffe04-7200-420d-b5d5-51df8078133a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.249875 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e3c1e94bd584bb78d9cf38ca66de1b9', 'request_id': 'req-d52ffe04-7200-420d-b5d5-51df8078133a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.259289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e3c1e94bd584bb78d9cf38ca66de1b9', 'request_id': 'req-d52ffe04-7200-420d-b5d5-51df8078133a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.272942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] The token cache is being invalidate because project 5e3c1e94bd584bb78d9cf38ca66de1b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:26:58.273228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.274456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d52ffe04-7200-420d-b5d5-51df8078133a tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.276143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1285/2599] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:26:58 2022] DELETE /identity/v3/projects/5e3c1e94bd584bb78d9cf38ca66de1b9 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:58.470842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:58.472629 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:58.536028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d94647b-85e5-43bb-9c1a-a8dc4faf515f tempest-VolumeQuotaClassesTest-1189857309 tempest-VolumeQuotaClassesTest-1189857309-project-admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:58.575217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:58.590403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:58.591021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:58.591297 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:58.591565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] PATH_INFO: `/v3/users/da2fbd030a3340fb8d548c209a7e375d` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:58.592674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da2fbd030a3340fb8d548c209a7e375d)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:58.594912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:58.643902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da2fbd030a3340fb8d548c209a7e375d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.644563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da2fbd030a3340fb8d548c209a7e375d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.651378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da2fbd030a3340fb8d548c209a7e375d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.652013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da2fbd030a3340fb8d548c209a7e375d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.663703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da2fbd030a3340fb8d548c209a7e375d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:58.676214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1315/2600] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:26:58 2022] DELETE /identity/v3/users/da2fbd030a3340fb8d548c209a7e375d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:58.680795 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:58.681075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:58.681345 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:58.700141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:26:58.701318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:58.758988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1286/2601] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:26:58.769527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:58.770946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:58.826449 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148c6682-949d-4f59-9780-a4f612dd2734 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:58.830747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:58.831340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:58.831614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:58.831884 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:58.840719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:58.842136 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:58.851909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1316/2602] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:26:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:26:59.361722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:59.363305 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:59.375672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815cebca-f277-4129-a52c-7c4af5cb0811 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:59.381684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:59.382480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:59.382767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:59.383027 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] PATH_INFO: `/v3/projects/59a7d0696ca94dbeafb8c8b4022bcbd4` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:59.384154 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a7d0696ca94dbeafb8c8b4022bcbd4)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:59.386565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:59.409422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:59.440488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a7d0696ca94dbeafb8c8b4022bcbd4', 'request_id': 'req-af12351e-47c3-4a8b-a062-f6c91284cc5f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.446171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a7d0696ca94dbeafb8c8b4022bcbd4', 'request_id': 'req-af12351e-47c3-4a8b-a062-f6c91284cc5f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.454271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a7d0696ca94dbeafb8c8b4022bcbd4', 'request_id': 'req-af12351e-47c3-4a8b-a062-f6c91284cc5f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.465898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] The token cache is being invalidate because project 59a7d0696ca94dbeafb8c8b4022bcbd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:26:59.466182 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.466911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.469788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1287/2603] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:26:59 2022] DELETE /identity/v3/projects/59a7d0696ca94dbeafb8c8b4022bcbd4 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:59.473226 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:59.483081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbee5949-54f9-4178-853f-3f82960853b0 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:59.539995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:59.540619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:59.540893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:59.541160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] PATH_INFO: `/v3/users/525b3a3aaf6f4970ba1208d215be098f` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:59.542294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=525b3a3aaf6f4970ba1208d215be098f)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:59.544187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:26:59.550114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:26:59.589100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '525b3a3aaf6f4970ba1208d215be098f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.589732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '525b3a3aaf6f4970ba1208d215be098f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.597472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '525b3a3aaf6f4970ba1208d215be098f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.598088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '525b3a3aaf6f4970ba1208d215be098f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.610258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-81f58f5e-c3f5-4c8b-9879-ff0189540482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '525b3a3aaf6f4970ba1208d215be098f'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:26:59.624267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1317/2604] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:26:59 2022] DELETE /identity/v3/users/525b3a3aaf6f4970ba1208d215be098f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:26:59.891302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:26:59.892771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:26:59.906341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12351e-47c3-4a8b-a062-f6c91284cc5f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:26:59.964137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:26:59.964791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:26:59.965075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:26:59.965349 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] PATH_INFO: `/v3/projects/744bd3ed52e04da4a0c99fb90a09b94f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:26:59.966494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] RBAC: Authorizing `identity:delete_project(project_id=744bd3ed52e04da4a0c99fb90a09b94f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:26:59.968379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.013884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '744bd3ed52e04da4a0c99fb90a09b94f', 'request_id': 'req-d2c40f37-f325-4667-aacc-5075827be691'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:00.019266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '744bd3ed52e04da4a0c99fb90a09b94f', 'request_id': 'req-d2c40f37-f325-4667-aacc-5075827be691'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:00.026177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '744bd3ed52e04da4a0c99fb90a09b94f', 'request_id': 'req-d2c40f37-f325-4667-aacc-5075827be691'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:00.037219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] The token cache is being invalidate because project 744bd3ed52e04da4a0c99fb90a09b94f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:00.037502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:00.038172 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d2c40f37-f325-4667-aacc-5075827be691 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:00.039646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1288/2605] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:26:59 2022] DELETE /identity/v3/projects/744bd3ed52e04da4a0c99fb90a09b94f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:00.044453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.044729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.045004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.065190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:00.066403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:00.123925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1318/2606] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:00.132944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.133221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.133489 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.152801 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:00.153999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:00.162323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1289/2607] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:00.166387 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.176704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaec1b-796c-47cf-b833-93c0079d8e33 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.181230 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.181813 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.182085 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.182391 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.183030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:27:00.183326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.184904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.196716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1319/2608] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:27:00.201360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.212960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46e31b9-50ab-4d1f-a819-14c362788c40 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.217520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.218107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.218415 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.219189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.219600 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.221475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.248269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1290/2609] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:00.252930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.262971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5d6f6-553c-49b4-8f70-77399dd20495 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.320527 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.321146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.321421 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.321689 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.322343 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.324233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.348992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1320/2610] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:00.350334 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.360808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02011f1e-de27-41ee-ad03-a149e1c97a8c admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.365380 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.366006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.366261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.366539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.367170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.368748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.370894 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:00.380211 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1291/2611] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:00.384273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.389432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:00.396635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde07948-1f1e-4c30-9d67-8cfa385dd930 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.402025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94bc5f3-3847-44fd-ac65-d792efaa73ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.402675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bc5f3-3847-44fd-ac65-d792efaa73ab admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.402949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bc5f3-3847-44fd-ac65-d792efaa73ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.403218 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94bc5f3-3847-44fd-ac65-d792efaa73ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.403852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bc5f3-3847-44fd-ac65-d792efaa73ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.405435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94bc5f3-3847-44fd-ac65-d792efaa73ab admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.415953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1321/2612] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:00.419976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.429761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d476fdfa-fd0b-4b90-bd48-8e9ce444896d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.434282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.434918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.435191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.435458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] PATH_INFO: `/v3/projects/0368cbf0af1942edb9f1a781f6605117/users/36085b2f44cb465c89426ccce12a62fd/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.450326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0368cbf0af1942edb9f1a781f6605117, user_id=36085b2f44cb465c89426ccce12a62fd, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.452932 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.466520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1292/2613] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:00 2022] PUT /identity/v3/projects/0368cbf0af1942edb9f1a781f6605117/users/36085b2f44cb465c89426ccce12a62fd/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:00.477025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.477497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.477606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.505294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] MFA Rules not processed for user `36085b2f44cb465c89426ccce12a62fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:00.506564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:00.570146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1322/2614] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:00.574853 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.646289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6c63b-65ec-4a28-b85d-7746a36d8366 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.651435 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.652131 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.652567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.652979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.653699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.655976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.684081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1293/2615] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/projects => generated 389 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:00.688331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.699505 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cc6c15-d17d-4cbd-a635-4393c9bdf7c3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.754817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.755420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.755691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.755962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.756581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.758515 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.779081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1323/2616] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/users => generated 301 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:00.783313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.794476 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e99152-d8ab-4750-9915-deafdbe14a78 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.799721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.800308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.800581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.800850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.801479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.803066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.813832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1294/2617] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:00.817944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.827817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb368330-bc7d-4c3e-8d3e-6746ed957c1d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.832712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.833345 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.833635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.833893 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] PATH_INFO: `/v3/projects/45e619c774d246afa65ba6325f504ca6/users/a1074610232e44ea86e15bb652a51a99/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.849185 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e619c774d246afa65ba6325f504ca6, user_id=a1074610232e44ea86e15bb652a51a99, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.852029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.865358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1324/2618] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:00 2022] PUT /identity/v3/projects/45e619c774d246afa65ba6325f504ca6/users/a1074610232e44ea86e15bb652a51a99/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:27:00.869441 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.879224 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8409d5-72c7-4b77-85f6-e60f7d2be26f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.934075 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794cfe4-e854-4b9b-ac1f-2cdbedd6a086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.934763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794cfe4-e854-4b9b-ac1f-2cdbedd6a086 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.935035 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794cfe4-e854-4b9b-ac1f-2cdbedd6a086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.935301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794cfe4-e854-4b9b-ac1f-2cdbedd6a086 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.935919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794cfe4-e854-4b9b-ac1f-2cdbedd6a086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.937539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8794cfe4-e854-4b9b-ac1f-2cdbedd6a086 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.946922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1295/2619] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:00.951140 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:00.961589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2892705a-78e3-4be9-87ed-c37a26b64c04 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:00.966126 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:00.966802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.967077 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.967349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] PATH_INFO: `/v3/projects/45e619c774d246afa65ba6325f504ca6/users/a1074610232e44ea86e15bb652a51a99/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:00.970019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e619c774d246afa65ba6325f504ca6, user_id=a1074610232e44ea86e15bb652a51a99, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:00.972659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:00.982657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1325/2620] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:00 2022] PUT /identity/v3/projects/45e619c774d246afa65ba6325f504ca6/users/a1074610232e44ea86e15bb652a51a99/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:27:00.993220 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:00.993510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:00.993787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.020871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] MFA Rules not processed for user `a1074610232e44ea86e15bb652a51a99`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:01.022065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.054264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:01.090227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1296/2621] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:01.102403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:01.104070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.139310 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:01.162886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c766c587-4e75-4220-9b9b-5fa1056597e0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:01.166466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aab22-4441-4f13-8505-2b47c3d3a9da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:01.167051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aab22-4441-4f13-8505-2b47c3d3a9da service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:01.167322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aab22-4441-4f13-8505-2b47c3d3a9da service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:01.167591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aab22-4441-4f13-8505-2b47c3d3a9da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.176964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aab22-4441-4f13-8505-2b47c3d3a9da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:01.178402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aab22-4441-4f13-8505-2b47c3d3a9da service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:01.187655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1326/2622] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:01.328722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:01.394890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4060e-7ba3-46c9-84ed-441afcaebe29 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:01.399367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:01.399980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:01.400254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:01.400523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] PATH_INFO: `/v3/users/36085b2f44cb465c89426ccce12a62fd` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.401618 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36085b2f44cb465c89426ccce12a62fd)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:01.403534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:01.448541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36085b2f44cb465c89426ccce12a62fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:01.449186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36085b2f44cb465c89426ccce12a62fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:01.461121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36085b2f44cb465c89426ccce12a62fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:01.461808 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36085b2f44cb465c89426ccce12a62fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:01.474614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36085b2f44cb465c89426ccce12a62fd'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:01.487141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1297/2623] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:01 2022] DELETE /identity/v3/users/36085b2f44cb465c89426ccce12a62fd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:01.491863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:01.492369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:01.492486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.510978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:01.512175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.571821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1327/2624] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:01.582295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:01.583694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.637896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:01.642517 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39339713-75f6-493b-870a-566880a49f50 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:01.647312 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:01.648112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:01.648567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:01.649019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.658120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:01.659616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:01.668791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1298/2625] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:01.940465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:01.942103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:01.954777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466f315f-1a63-46b9-8fd9-2deefaa749bc None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:01.959333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:01.959942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:01.960215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:01.960483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] PATH_INFO: `/v3/projects/0368cbf0af1942edb9f1a781f6605117` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:01.961581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0368cbf0af1942edb9f1a781f6605117)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:01.963964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.010283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0368cbf0af1942edb9f1a781f6605117', 'request_id': 'req-99032919-91ba-49a8-ba37-4835bdd98737'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.016992 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0368cbf0af1942edb9f1a781f6605117', 'request_id': 'req-99032919-91ba-49a8-ba37-4835bdd98737'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.022714 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0368cbf0af1942edb9f1a781f6605117', 'request_id': 'req-99032919-91ba-49a8-ba37-4835bdd98737'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.035358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] The token cache is being invalidate because project 0368cbf0af1942edb9f1a781f6605117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:02.035649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.036275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.037834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1328/2626] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:01 2022] DELETE /identity/v3/projects/0368cbf0af1942edb9f1a781f6605117 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:27:02.041800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.051746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4797d6-cc0e-43bf-a36b-4d293b4469b1 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.106087 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.106729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.107001 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.107268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] PATH_INFO: `/v3/users/a1074610232e44ea86e15bb652a51a99` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.108355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1074610232e44ea86e15bb652a51a99)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:02.110947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.155539 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1074610232e44ea86e15bb652a51a99'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.156776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1074610232e44ea86e15bb652a51a99'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.163673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1074610232e44ea86e15bb652a51a99'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.164285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1074610232e44ea86e15bb652a51a99'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.176705 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-c8575f41-069a-4d26-9cda-3eb756430985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1074610232e44ea86e15bb652a51a99'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.189137 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1299/2627] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:02 2022] DELETE /identity/v3/users/a1074610232e44ea86e15bb652a51a99 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:02.466033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.467625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:02.481291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99032919-91ba-49a8-ba37-4835bdd98737 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.538875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.539522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.539756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.540021 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] PATH_INFO: `/v3/projects/45e619c774d246afa65ba6325f504ca6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.541105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e619c774d246afa65ba6325f504ca6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:02.543030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.544398 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:02.589966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e619c774d246afa65ba6325f504ca6', 'request_id': 'req-75dc4ae3-a342-42a3-8aab-62032ebb1696'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.596389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e619c774d246afa65ba6325f504ca6', 'request_id': 'req-75dc4ae3-a342-42a3-8aab-62032ebb1696'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.601882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e619c774d246afa65ba6325f504ca6', 'request_id': 'req-75dc4ae3-a342-42a3-8aab-62032ebb1696'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.613886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] The token cache is being invalidate because project 45e619c774d246afa65ba6325f504ca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:02.614172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.614827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-75dc4ae3-a342-42a3-8aab-62032ebb1696 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:02.620270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1329/2628] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:02 2022] DELETE /identity/v3/projects/45e619c774d246afa65ba6325f504ca6 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:02.625857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.626348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.626542 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.646761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:02.647957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:02.707731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1300/2629] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:02.716538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.716763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.717039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.738210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:02.739436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:02.747275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1330/2630] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:02.752516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.762223 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d706ee-d7df-42b2-b7f9-02fe47fc8494 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.766974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.767569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.767848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.768124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.768763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:27:02.769067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:02.770700 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.777307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:02.783080 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1301/2631] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:27:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:27:02.787964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.799809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ead45b-06f8-4981-acc4-52207e3ae162 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.804986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.805621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.805934 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.806293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.806988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:02.809176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.839593 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1331/2632] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:02 2022] POST /identity/v3/projects => generated 409 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:02.844036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.853861 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d32052-87ed-47ff-9c0f-76ee917fcba2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.910857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.911455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.911732 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.912002 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.912617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:02.914521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.939041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1302/2633] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:02 2022] POST /identity/v3/users => generated 305 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:02.943658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.956042 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b872890-4845-4f97-bdf5-fec1c0093ae5 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.960692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.961288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.961562 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.961830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.962514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:02.964099 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:02.978459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1332/2634] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:27:02.980471 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:02.991242 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc3b5b9-cffa-409b-9a5f-c36e989fd72a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:02.996267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b58a1-6a34-494c-ab9d-b43204b8e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:02.996900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b58a1-6a34-494c-ab9d-b43204b8e8fc admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:02.997375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b58a1-6a34-494c-ab9d-b43204b8e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:02.997653 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b58a1-6a34-494c-ab9d-b43204b8e8fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:02.998324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b58a1-6a34-494c-ab9d-b43204b8e8fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.000522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910b58a1-6a34-494c-ab9d-b43204b8e8fc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.012592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1303/2635] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:03.018100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.028539 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5a6517-ba9d-42fb-9d95-764d0dd088db admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.033274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.033918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.034191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.034494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] PATH_INFO: `/v3/projects/699ead2b76704c759bb36b0333c90176/users/b8c657cfc6b143aab86f50a68f11c649/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.050238 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] RBAC: Authorizing `identity:create_grant(project_id=699ead2b76704c759bb36b0333c90176, user_id=b8c657cfc6b143aab86f50a68f11c649, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.052863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.067510 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1333/2636] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:03 2022] PUT /identity/v3/projects/699ead2b76704c759bb36b0333c90176/users/b8c657cfc6b143aab86f50a68f11c649/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:03.079617 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.079908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.080184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.108181 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] MFA Rules not processed for user `b8c657cfc6b143aab86f50a68f11c649`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:03.109938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:03.171578 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1304/2637] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:03.176739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.237199 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96c4bc1-59aa-4b2c-a7a3-f63fddb1629c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.241767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.242393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.242672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.242942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.243569 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.245451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.271168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1334/2638] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:03 2022] POST /identity/v3/projects => generated 407 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:03.275419 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.286309 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1281d8b9-7cc5-4b95-b493-656335b2e134 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.341921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.342571 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.342848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.343118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.344508 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.346454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.359963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:03.371328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1305/2639] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:03 2022] POST /identity/v3/users => generated 310 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:03.375799 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.386222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c26c4d-516c-4fe6-ba74-81870ac8c169 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.391033 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.391637 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.391915 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.392187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.392856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.394479 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.399811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:03.407924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1335/2640] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:03.412117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.422734 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04177d0-856f-49e6-b7d3-dbdd08ba077a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.427687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.428485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.428857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.429236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] PATH_INFO: `/v3/projects/4adcf670e3064961967caabf12e68ef1/users/89c857b4fe054ba1b02ec977427ed34c/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.444951 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4adcf670e3064961967caabf12e68ef1, user_id=89c857b4fe054ba1b02ec977427ed34c, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.448138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.467197 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1306/2641] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:03 2022] PUT /identity/v3/projects/4adcf670e3064961967caabf12e68ef1/users/89c857b4fe054ba1b02ec977427ed34c/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:03.472418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.482771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e3548-3c12-435d-b3ac-f7998734f6b6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.539171 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c9793-e33c-4ba2-82eb-a8f2d1e5afbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.539798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c9793-e33c-4ba2-82eb-a8f2d1e5afbf admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.540027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c9793-e33c-4ba2-82eb-a8f2d1e5afbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.540293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c9793-e33c-4ba2-82eb-a8f2d1e5afbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.540917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c9793-e33c-4ba2-82eb-a8f2d1e5afbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.542532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811c9793-e33c-4ba2-82eb-a8f2d1e5afbf admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.552060 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1336/2642] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:03 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:03.556169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.566318 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d31207-f819-4945-8ea1-6d8991f0e32a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.571077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.571720 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.571994 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.572268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] PATH_INFO: `/v3/projects/4adcf670e3064961967caabf12e68ef1/users/89c857b4fe054ba1b02ec977427ed34c/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.574157 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4adcf670e3064961967caabf12e68ef1, user_id=89c857b4fe054ba1b02ec977427ed34c, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.576787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.587486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1307/2643] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:03 2022] PUT /identity/v3/projects/4adcf670e3064961967caabf12e68ef1/users/89c857b4fe054ba1b02ec977427ed34c/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:03.600123 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.600406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.600679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.629451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] MFA Rules not processed for user `89c857b4fe054ba1b02ec977427ed34c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:03.630761 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:03.697410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1337/2644] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:03.708032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.709806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:03.766976 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d63107-71b1-4849-87df-b238c977308a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.770486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.771069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.771343 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.771611 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:03.780257 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:03.781673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:03.790800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1308/2645] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:03.939423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:03.948819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737fa0c1-95af-4957-b40d-9da23bc696a3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:03.952349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a700d4-6fb6-4d3c-b7fd-8cee3f39f59f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:03.953571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a700d4-6fb6-4d3c-b7fd-8cee3f39f59f service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:03.953832 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a700d4-6fb6-4d3c-b7fd-8cee3f39f59f service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:03.954101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a700d4-6fb6-4d3c-b7fd-8cee3f39f59f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:04.010132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a700d4-6fb6-4d3c-b7fd-8cee3f39f59f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:04.011604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a700d4-6fb6-4d3c-b7fd-8cee3f39f59f service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:04.021236 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1338/2646] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:03 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:11.371410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:11.445043 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3492d55f-77bc-46c7-8bdc-df642f6618e0 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:11.450530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:11.451229 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:11.451726 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:11.452216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] PATH_INFO: `/v3/users/b8c657cfc6b143aab86f50a68f11c649` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:11.454957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8c657cfc6b143aab86f50a68f11c649)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:11.458240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:11.510554 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8c657cfc6b143aab86f50a68f11c649'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:11.511259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8c657cfc6b143aab86f50a68f11c649'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:11.519971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8c657cfc6b143aab86f50a68f11c649'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:11.520632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8c657cfc6b143aab86f50a68f11c649'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:11.526155 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:11.535825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8c657cfc6b143aab86f50a68f11c649'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:11.550483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1309/2647] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:11 2022] DELETE /identity/v3/users/b8c657cfc6b143aab86f50a68f11c649 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:11.556004 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:11.556500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:11.556613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:11.576906 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:11.579160 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:11.592046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:11.643434 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1339/2648] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:11.653692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:11.655268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:11.715942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2127af5b-3fbb-49f2-8170-6abacf0c2845 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:11.719743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:11.720398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:11.720668 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:11.720936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:11.730637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:11.732056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:11.742082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1310/2649] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:12.026745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:12.028145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:12.040131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3101cf-5b41-40ff-9dd0-55a0ca0cbb0b None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:12.044705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:12.045336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.045604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.045868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] PATH_INFO: `/v3/projects/699ead2b76704c759bb36b0333c90176` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.047543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699ead2b76704c759bb36b0333c90176)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:12.049538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:12.097138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699ead2b76704c759bb36b0333c90176', 'request_id': 'req-48a4efe4-084d-4993-9c35-ff91a214bcf7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.103354 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699ead2b76704c759bb36b0333c90176', 'request_id': 'req-48a4efe4-084d-4993-9c35-ff91a214bcf7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.109472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699ead2b76704c759bb36b0333c90176', 'request_id': 'req-48a4efe4-084d-4993-9c35-ff91a214bcf7'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.121864 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] The token cache is being invalidate because project 699ead2b76704c759bb36b0333c90176 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:12.122149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.122792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.124258 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1340/2650] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:12 2022] DELETE /identity/v3/projects/699ead2b76704c759bb36b0333c90176 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:12.128308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:12.137776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157ecbfc-90e7-456f-aead-12e22e1fd10d service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:12.194121 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:12.194784 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.195063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.195335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] PATH_INFO: `/v3/users/89c857b4fe054ba1b02ec977427ed34c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.196437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c857b4fe054ba1b02ec977427ed34c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:12.198378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:12.245119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c857b4fe054ba1b02ec977427ed34c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.245686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c857b4fe054ba1b02ec977427ed34c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.253160 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c857b4fe054ba1b02ec977427ed34c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.253788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c857b4fe054ba1b02ec977427ed34c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.265942 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-06dbc248-b9a7-4b57-9bd8-c9350ec5fef2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c857b4fe054ba1b02ec977427ed34c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.278847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1311/2651] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:12 2022] DELETE /identity/v3/users/89c857b4fe054ba1b02ec977427ed34c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:12.637332 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:12.639169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:12.654896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4efe4-084d-4993-9c35-ff91a214bcf7 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:12.712901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:12.713594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.713912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.714261 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] PATH_INFO: `/v3/projects/4adcf670e3064961967caabf12e68ef1` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.715523 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=4adcf670e3064961967caabf12e68ef1)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:12.717603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:12.761560 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:12.764701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4adcf670e3064961967caabf12e68ef1', 'request_id': 'req-aa13bf28-d40e-49c7-ba1d-210fc27551ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.772127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4adcf670e3064961967caabf12e68ef1', 'request_id': 'req-aa13bf28-d40e-49c7-ba1d-210fc27551ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.777572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4adcf670e3064961967caabf12e68ef1', 'request_id': 'req-aa13bf28-d40e-49c7-ba1d-210fc27551ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.789573 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] The token cache is being invalidate because project 4adcf670e3064961967caabf12e68ef1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:12.789798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.790438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-aa13bf28-d40e-49c7-ba1d-210fc27551ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:12.791890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1341/2652] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:12 2022] DELETE /identity/v3/projects/4adcf670e3064961967caabf12e68ef1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:12.797297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.797881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.797881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.805352 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:12.819992 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:12.821205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:12.878556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1312/2653] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:12.886438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.886721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.887020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.906831 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:12.908023 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:12.915876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1342/2654] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:12.919971 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:12.932491 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d9ff1-09d0-4007-a60e-87020e0464e4 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:12.939218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:12.939862 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.940246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.940549 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.941205 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:27:12.941512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:12.943136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:12.955271 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1313/2655] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:27:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:27:12.959717 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:12.969718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74df87c4-c55c-48e0-bf75-7aef4bdedcb7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:12.976102 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:12.976900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:12.977172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:12.977441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:12.978491 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:12.980656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:13.011442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1343/2656] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:12 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:13.015900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:13.025860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ebaba8-4c6a-434c-8241-11f2f14a506b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:13.081047 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:13.081814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:13.082091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:13.082391 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:13.083028 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:13.084910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:13.106092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1314/2657] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:13 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:13.111071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:13.121551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fc9dfe-3082-4f94-b09a-f732aa9511d3 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:13.126113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15488375-5937-45e0-9747-54f81add801d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:13.126736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15488375-5937-45e0-9747-54f81add801d admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:13.127012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15488375-5937-45e0-9747-54f81add801d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:13.127280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15488375-5937-45e0-9747-54f81add801d admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:13.127909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15488375-5937-45e0-9747-54f81add801d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:13.129483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15488375-5937-45e0-9747-54f81add801d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:13.139067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1344/2658] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:13.142826 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:13.152414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f28fd86-a16e-4af4-a2c7-12ae0d547492 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:13.157484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b088d77-dd08-4407-8aae-1b4a29f6e14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:13.158055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b088d77-dd08-4407-8aae-1b4a29f6e14e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:13.158421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b088d77-dd08-4407-8aae-1b4a29f6e14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:13.158692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b088d77-dd08-4407-8aae-1b4a29f6e14e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:13.159315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b088d77-dd08-4407-8aae-1b4a29f6e14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:13.160871 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b088d77-dd08-4407-8aae-1b4a29f6e14e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:13.171876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1315/2659] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:13.175987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15488375-5937-45e0-9747-54f81add801d admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:13.186220 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15488375-5937-45e0-9747-54f81add801d admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:13.191177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:13.191819 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:13.192098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:13.192397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] PATH_INFO: `/v3/projects/48ebfff064b4471cb362fed132761729/users/2fdf0785b93c441b8f20780967a5c3c3/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:13.207057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48ebfff064b4471cb362fed132761729, user_id=2fdf0785b93c441b8f20780967a5c3c3, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:13.209685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:13.223264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1345/2660] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:13 2022] PUT /identity/v3/projects/48ebfff064b4471cb362fed132761729/users/2fdf0785b93c441b8f20780967a5c3c3/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:27:13.228256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f387e-5d02-4371-8ad5-e0354d5d6beb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:13.228540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f387e-5d02-4371-8ad5-e0354d5d6beb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:13.228810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407f387e-5d02-4371-8ad5-e0354d5d6beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:13.247893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-407f387e-5d02-4371-8ad5-e0354d5d6beb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:13.249077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-407f387e-5d02-4371-8ad5-e0354d5d6beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:13.308064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1316/2661] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:13.317741 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:13.319429 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:13.383174 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ccac54-254d-4580-ade4-ba1e55f7ba42 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:13.386938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:13.387551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:13.387848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:13.388115 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:13.398038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:13.399508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:13.410929 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1346/2662] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:14.128282 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.128777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.128873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.147900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:14.158293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] MFA Rules not processed for user `2fdf0785b93c441b8f20780967a5c3c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:14.159488 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:14.224825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1317/2663] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:14.229319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.239453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30afe05-f6c5-4d8e-94b7-a165770e19a5 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.243973 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.244566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.244823 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.245089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.245704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:14.247673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:14.275600 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1347/2664] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:14 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:27:14.279796 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.290819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7832e7d2-c720-4797-b8b3-2dfb148c2bf5 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.349244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.349852 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.350124 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.350420 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.351054 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:14.352950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:14.375510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1318/2665] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:14 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:27:14.379721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.390264 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcb0332-b64e-4b8f-bdda-46f373e50b4b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.394431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.395016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.395289 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.395555 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.396190 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:14.397854 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:14.408980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1348/2666] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:14.413011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.423566 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734fa682-b325-4091-8ffb-9ce905b324ca admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.428083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.428721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.428997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.429276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] PATH_INFO: `/v3/projects/1fd090047caf4d288367f9a422fd61d0/users/491efb9e82684337a588392c6d397406/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.443581 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd090047caf4d288367f9a422fd61d0, user_id=491efb9e82684337a588392c6d397406, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:14.446228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:14.458828 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1319/2667] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:14 2022] PUT /identity/v3/projects/1fd090047caf4d288367f9a422fd61d0/users/491efb9e82684337a588392c6d397406/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:14.462794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.473482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469eb94b-3c4c-4172-9584-a9d1016c2852 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.510938 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:14.531144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd98c52-d96b-4a8e-bb9c-162738fcc25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.532283 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98c52-d96b-4a8e-bb9c-162738fcc25e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.532557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98c52-d96b-4a8e-bb9c-162738fcc25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.532828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd98c52-d96b-4a8e-bb9c-162738fcc25e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.533469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd98c52-d96b-4a8e-bb9c-162738fcc25e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:14.535088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd98c52-d96b-4a8e-bb9c-162738fcc25e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:14.544548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1349/2668] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:14.549313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:14.559386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f1f91-10f7-49d2-8200-6880f3b39884 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:14.564036 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:14.564672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:14.564945 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:14.565215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] PATH_INFO: `/v3/projects/1fd090047caf4d288367f9a422fd61d0/users/491efb9e82684337a588392c6d397406/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:14.567140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fd090047caf4d288367f9a422fd61d0, user_id=491efb9e82684337a588392c6d397406, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:14.569741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:14.579846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1320/2669] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:14 2022] PUT /identity/v3/projects/1fd090047caf4d288367f9a422fd61d0/users/491efb9e82684337a588392c6d397406/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:15.189908 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:15.190337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:15.190631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:15.218249 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] MFA Rules not processed for user `491efb9e82684337a588392c6d397406`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:15.219633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:15.292474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1350/2670] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:15 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:15.305927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:15.308194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:15.370760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46320416-beec-4674-9a9d-ecdcffa8c367 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:15.375250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aeabdd-4c47-4d53-8d01-327e9c808f06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:15.375935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aeabdd-4c47-4d53-8d01-327e9c808f06 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:15.376210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aeabdd-4c47-4d53-8d01-327e9c808f06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:15.376475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aeabdd-4c47-4d53-8d01-327e9c808f06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:15.380613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:15.434672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aeabdd-4c47-4d53-8d01-327e9c808f06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:15.436123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aeabdd-4c47-4d53-8d01-327e9c808f06 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:15.445371 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1321/2671] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:16.795921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:16.797300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:16.809377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e83d-6dba-4484-a2a9-a03119a11208 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:16.813422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:16.814011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:16.814313 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:16.814582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:16.823274 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:16.824691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:16.834901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1351/2672] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:16 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:17.760288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:17.760785 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:17.761077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:17.782088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:17.783377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:17.839713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1322/2673] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:17.887280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:17.888745 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:17.960784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e39bf6d-e15f-449f-8db9-a0f29ada6fea service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:17.965855 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:17.966480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:17.966759 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:17.967029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:17.975690 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:17.977272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:17.986149 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1352/2674] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:27:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:27:27.833020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=50 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:27.982623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=83 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:30.526844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:30.591444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cf4290-5b46-4d60-87ee-068b3c058672 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:30.596308 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:30.596936 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:30.597213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:30.597481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] PATH_INFO: `/v3/users/2fdf0785b93c441b8f20780967a5c3c3` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:30.598714 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fdf0785b93c441b8f20780967a5c3c3)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:30.600606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:30.649329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fdf0785b93c441b8f20780967a5c3c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:30.649972 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fdf0785b93c441b8f20780967a5c3c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:30.656710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fdf0785b93c441b8f20780967a5c3c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:30.657328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fdf0785b93c441b8f20780967a5c3c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:30.669372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fdf0785b93c441b8f20780967a5c3c3'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:30.682573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1323/2675] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:30 2022] DELETE /identity/v3/users/2fdf0785b93c441b8f20780967a5c3c3 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:27:31.075059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.091446 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67460bf9-f1d6-47e7-9ba8-394bbcb6f3f1 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.155382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.156657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.156991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.157285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] PATH_INFO: `/v3/projects/48ebfff064b4471cb362fed132761729` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.158465 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=48ebfff064b4471cb362fed132761729)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:31.160357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:31.208594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48ebfff064b4471cb362fed132761729', 'request_id': 'req-f993b81e-5bad-4058-9d7e-facd5f42a3fd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.214640 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48ebfff064b4471cb362fed132761729', 'request_id': 'req-f993b81e-5bad-4058-9d7e-facd5f42a3fd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.221766 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48ebfff064b4471cb362fed132761729', 'request_id': 'req-f993b81e-5bad-4058-9d7e-facd5f42a3fd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.236556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] The token cache is being invalidate because project 48ebfff064b4471cb362fed132761729 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:31.236975 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.237683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.239286 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1353/2676] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:31 2022] DELETE /identity/v3/projects/48ebfff064b4471cb362fed132761729 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:31.243411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.244842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.257753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a1a5-fdc7-4910-a8cc-2290dc2dca1b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.317328 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.318403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.319077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.319470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] PATH_INFO: `/v3/users/491efb9e82684337a588392c6d397406` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.320744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] RBAC: Authorizing `identity:delete_user(user_id=491efb9e82684337a588392c6d397406)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:31.322622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:31.346658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:31.373961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491efb9e82684337a588392c6d397406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.374766 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491efb9e82684337a588392c6d397406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.381958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491efb9e82684337a588392c6d397406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.382752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491efb9e82684337a588392c6d397406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.395084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491efb9e82684337a588392c6d397406'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.397574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.399132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.408667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1324/2677] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:31 2022] DELETE /identity/v3/users/491efb9e82684337a588392c6d397406 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:31.459010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f993b81e-5bad-4058-9d7e-facd5f42a3fd admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.505472 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:31.511918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.512724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.513370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.513734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.516705 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.526384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:31.527835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fea581-0a17-4fb6-ad4e-a92068e356f5 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:31.536886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1354/2678] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:27:31.584402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.585950 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.677553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d79ba-42a6-4f21-a71d-035fcaa5291c None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.677846 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d79ba-42a6-4f21-a71d-035fcaa5291c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.678112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d79ba-42a6-4f21-a71d-035fcaa5291c None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.680481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1355/2679] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:27:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:27:31.683313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da35cc0-faf7-455c-8b13-761040b000df admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.686426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.686767 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.686969 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.689311 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.690093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.690503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.690874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] PATH_INFO: `/v3/projects/1fd090047caf4d288367f9a422fd61d0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.692228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fd090047caf4d288367f9a422fd61d0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:31.694472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:31.709966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:31.711820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.753437 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fd090047caf4d288367f9a422fd61d0', 'request_id': 'req-25600163-2b2c-4ac3-9918-1d4701d40e47'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.761015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fd090047caf4d288367f9a422fd61d0', 'request_id': 'req-25600163-2b2c-4ac3-9918-1d4701d40e47'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.772556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fd090047caf4d288367f9a422fd61d0', 'request_id': 'req-25600163-2b2c-4ac3-9918-1d4701d40e47'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.786604 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] The token cache is being invalidate because project 1fd090047caf4d288367f9a422fd61d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:31.786922 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.787530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-25600163-2b2c-4ac3-9918-1d4701d40e47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:31.789108 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1325/2680] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:31 2022] DELETE /identity/v3/projects/1fd090047caf4d288367f9a422fd61d0 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:31.792419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1356/2681] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:31.793997 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723421f-9286-4b18-8b94-f0c7842bef7e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.794303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723421f-9286-4b18-8b94-f0c7842bef7e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.794576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a723421f-9286-4b18-8b94-f0c7842bef7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.804860 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.806293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.818834 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a723421f-9286-4b18-8b94-f0c7842bef7e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:31.820249 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a723421f-9286-4b18-8b94-f0c7842bef7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.848253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:31.871173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca714ca-8653-4cd4-8212-c7a2c0290283 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.888442 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1326/2682] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:31.896570 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.896854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.897129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.923399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:31.924729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.930139 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.930646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.930918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.931321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.933010 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:31.935898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1327/2683] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:31.940255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.951657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314b437-5dee-4911-bfe3-dc32d6028bce None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.956895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:31.957493 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:31.958050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:31.958050 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:31.958690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:27:31.958985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:31.960608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:31.975848 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1328/2684] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:27:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 04 10:27:31.981055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:31.996132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806e8e19-e69f-485e-a231-a607cd5d2869 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:31.997129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:32.002655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.003430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.003913 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.004284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.004987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.007374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.023517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.025563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d48537-5911-404d-b891-5db27a6f2609 service glance] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.040741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1329/2685] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:31 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:32.040964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1357/2686] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:27:32.043164 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.110778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73f0505-f963-412d-9826-06b6182cdf6f admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.111553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47243e2-d8be-4551-8986-67cdc3e87bf3 None None] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.112036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47243e2-d8be-4551-8986-67cdc3e87bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.112175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47243e2-d8be-4551-8986-67cdc3e87bf3 None None] PATH_INFO: `/` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.113469 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1358/2687] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:27:32.116065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.116735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.117079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.117381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.117868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.117999 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.118163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.118462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.119943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.139015 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] MFA Rules not processed for user `0e568115c67243d3904359b84e0a790e`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:32.140737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:32.146037 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1330/2688] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:32.147044 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.157295 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36f1c49-4c51-4990-86f4-ff7f6c2abeb8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.162577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.164226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.165146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.165631 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.166979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.170603 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.181056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1331/2689] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:32.184968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.194530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139f0ca3-1c90-4ba2-ac3b-78c74e2b386d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.196240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1359/2690] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:32.199199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.199781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.200060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.200332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.200960 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.202673 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.208740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.210101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:32.217060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1332/2691] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:27:32.217988 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.221545 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556eb59c-7da1-4c57-a978-cdcfa5db0a90 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.225196 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b93e79-e77a-43d5-bfe3-bfa1645ade24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.225780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93e79-e77a-43d5-bfe3-bfa1645ade24 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.226057 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93e79-e77a-43d5-bfe3-bfa1645ade24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.226364 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b93e79-e77a-43d5-bfe3-bfa1645ade24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.228366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f5f8bd-028a-4374-9efa-ecf2a894005b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.232934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7445641-06d1-4eab-9d38-6244de78ae8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.233563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7445641-06d1-4eab-9d38-6244de78ae8c admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.233716 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b93e79-e77a-43d5-bfe3-bfa1645ade24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.233820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7445641-06d1-4eab-9d38-6244de78ae8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.234264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7445641-06d1-4eab-9d38-6244de78ae8c admin admin] PATH_INFO: `/v3/projects/de56a7f061054cd68c1f676e8fd35d71/users/4a744e75fc2e40909618999d05ec6d0f/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.235119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b93e79-e77a-43d5-bfe3-bfa1645ade24 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.242892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1360/2692] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:27:32.248723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7445641-06d1-4eab-9d38-6244de78ae8c admin admin] RBAC: Authorizing `identity:create_grant(project_id=de56a7f061054cd68c1f676e8fd35d71, user_id=4a744e75fc2e40909618999d05ec6d0f, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.251722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7445641-06d1-4eab-9d38-6244de78ae8c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.265100 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1333/2693] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:32 2022] PUT /identity/v3/projects/de56a7f061054cd68c1f676e8fd35d71/users/4a744e75fc2e40909618999d05ec6d0f/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:32.277291 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.277657 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.278005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.308342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] MFA Rules not processed for user `4a744e75fc2e40909618999d05ec6d0f`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:32.309594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:32.330799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95a8e5-dd4f-4639-ab14-be49fdfd6f6e None None] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.331167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95a8e5-dd4f-4639-ab14-be49fdfd6f6e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.331417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea95a8e5-dd4f-4639-ab14-be49fdfd6f6e None None] PATH_INFO: `/` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.332772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1334/2694] 10.222.0.13 () {54 vars in 921 bytes} [Tue Oct 4 10:27:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 04 10:27:32.338003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.338382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.338657 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.360046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] MFA Rules not processed for user `a8514e96615c4785aefde8261120afe3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:32.361481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:32.378991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1361/2695] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:32.383565 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.392565 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:32.438083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1335/2696] 10.222.0.13 () {58 vars in 1018 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:32.445649 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed4e623-238b-43dc-8c7d-9c97d34ac8fe None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.448694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.450317 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.450921 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.451198 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.451468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.452133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.454635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.464271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=100 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:32.482598 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1362/2697] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:27:32.488825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.506410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d1bd1-750c-45b5-af65-e7455bbc6460 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.551572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e1840a-fca1-4eea-8b96-9beb9815b7b0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.555797 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.556404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.556557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.556708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.556938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.557114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.557395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.557669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.558326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.560254 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.583425 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1363/2698] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:27:32.587782 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.597858 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cc1e5b-5146-42e2-8ab4-67460905ece6 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.602432 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.603014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.603293 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.603558 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.604183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.605927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.615709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1364/2699] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:27:32.619643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.630164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d543a12-b257-4825-815f-1f784a0b83ae admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.634032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.635204 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd301d87-25ee-442a-bec5-bf1095cd6611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.635620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.635798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301d87-25ee-442a-bec5-bf1095cd6611 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.636075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301d87-25ee-442a-bec5-bf1095cd6611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.636347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd301d87-25ee-442a-bec5-bf1095cd6611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.636984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd301d87-25ee-442a-bec5-bf1095cd6611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.638612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd301d87-25ee-442a-bec5-bf1095cd6611 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.647269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1336/2700] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:27:32.650471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1365/2701] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:27:32.654780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.665689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5636c4-8198-497c-90d2-188c6dc13a9c service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.670360 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.671084 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.671369 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.672167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] PATH_INFO: `/v3/projects/dbf83ee83dc041e8a09dfd785c912a54/users/add32d3fd52b472fb32f4bb5b12687be/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.687174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbf83ee83dc041e8a09dfd785c912a54, user_id=add32d3fd52b472fb32f4bb5b12687be, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.690378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.693326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:32.705306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1337/2702] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:32 2022] PUT /identity/v3/projects/dbf83ee83dc041e8a09dfd785c912a54/users/add32d3fd52b472fb32f4bb5b12687be/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:27:32.715636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.715927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.716246 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.744347 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] MFA Rules not processed for user `add32d3fd52b472fb32f4bb5b12687be`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:32.745563 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:32.812930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1366/2703] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:32.817411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.885403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be8aabf-42c5-48e6-b5fc-a71a2d085212 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.890113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:32.890919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:32.891195 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:32.891469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:32.892130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:32.894250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:32.924143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1338/2704] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/projects => generated 383 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:32.928422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:32.942396 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049627d4-045a-4d88-ab78-c111bce0e4e3 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:32.993891 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:33.012262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:33.012875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.013168 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.013436 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.014260 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:33.016376 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:33.041924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1367/2705] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:32 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:33.043130 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:33.053895 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f344afcc-8fbb-4752-8247-348fb6d1504b admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:33.058682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:33.059299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.059572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.059846 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.060472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:33.062060 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:33.072949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1339/2706] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:33.077068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:33.087114 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90eb5b98-f861-4d47-9673-c63909e906a1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:33.091722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:33.092371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.092646 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.092923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] PATH_INFO: `/v3/projects/269d0df4445047fb98f0870501b9d5a8/users/4350985e648e4da38d01fc251193e4a0/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.108612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269d0df4445047fb98f0870501b9d5a8, user_id=4350985e648e4da38d01fc251193e4a0, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:33.112337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:33.126820 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1368/2707] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:33 2022] PUT /identity/v3/projects/269d0df4445047fb98f0870501b9d5a8/users/4350985e648e4da38d01fc251193e4a0/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:33.131246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:33.142367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36809fc-425a-4ae0-beb7-6d8a27d6db9d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:33.198774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c3fb3-c4ad-40e2-87d4-420db4fb4e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:33.199451 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3fb3-c4ad-40e2-87d4-420db4fb4e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.199648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3fb3-c4ad-40e2-87d4-420db4fb4e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.199921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c3fb3-c4ad-40e2-87d4-420db4fb4e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.200551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3fb3-c4ad-40e2-87d4-420db4fb4e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:33.202219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c3fb3-c4ad-40e2-87d4-420db4fb4e6e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:33.213092 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1340/2708] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:27:33.216633 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:33.227494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf8c7d-2830-48fd-aae7-8a44bc13a1a4 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:33.232091 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:33.232735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.233007 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.233275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] PATH_INFO: `/v3/projects/269d0df4445047fb98f0870501b9d5a8/users/4350985e648e4da38d01fc251193e4a0/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.236528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=269d0df4445047fb98f0870501b9d5a8, user_id=4350985e648e4da38d01fc251193e4a0, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:33.239414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:33.251534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1369/2709] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:33 2022] PUT /identity/v3/projects/269d0df4445047fb98f0870501b9d5a8/users/4350985e648e4da38d01fc251193e4a0/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:33.261614 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.261896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.262167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.289984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] MFA Rules not processed for user `4350985e648e4da38d01fc251193e4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:33.291255 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:33.346610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:33.367049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1341/2710] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:33 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:33.379420 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:33.381111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:33.438240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb02f87-c100-4532-91fb-88117dd89d53 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:33.441835 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87979394-dcad-4d07-94ae-68ad9b15064d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:33.442473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87979394-dcad-4d07-94ae-68ad9b15064d service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:33.442753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87979394-dcad-4d07-94ae-68ad9b15064d service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:33.443029 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87979394-dcad-4d07-94ae-68ad9b15064d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:33.502580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87979394-dcad-4d07-94ae-68ad9b15064d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:33.504012 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87979394-dcad-4d07-94ae-68ad9b15064d service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:33.513336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1370/2711] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:33 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:35.021985 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:35.032478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be975e-dc68-4d29-8f64-f9859e451a40 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:35.036254 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:35.036876 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:35.037193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:35.037504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:35.099288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:35.100622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:35.109774 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1342/2712] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:35.402468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:35.402930 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:35.403284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:35.424762 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:35.426022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:35.476459 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:35.485984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1371/2713] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:27:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:35.498911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:35.500353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:35.576077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d8af8-92fe-43c8-9cf1-da1af382d69c service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:35.581272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:35.581874 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:35.582152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:35.582458 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:35.592184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:35.594029 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:35.602660 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1343/2714] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:27:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:27:35.689370 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:35.701268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d816b21-b614-43d3-94a9-4ce62a002672 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:35.705869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fad4fb-6bc2-4e00-acf2-edcc28aa3845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:35.706870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fad4fb-6bc2-4e00-acf2-edcc28aa3845 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:35.707389 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fad4fb-6bc2-4e00-acf2-edcc28aa3845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:35.707881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fad4fb-6bc2-4e00-acf2-edcc28aa3845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:35.724447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fad4fb-6bc2-4e00-acf2-edcc28aa3845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:35.726457 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fad4fb-6bc2-4e00-acf2-edcc28aa3845 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:35.762427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1372/2715] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:35 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:39.795754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:39.865070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c6191-b8d9-429f-9cfd-5570d44dbfc4 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:39.870425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:39.871062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:39.871456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:39.871728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] PATH_INFO: `/v3/users/4a744e75fc2e40909618999d05ec6d0f` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:39.872854 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a744e75fc2e40909618999d05ec6d0f)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:39.874800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:39.898941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:39.930749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a744e75fc2e40909618999d05ec6d0f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:39.931530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a744e75fc2e40909618999d05ec6d0f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:39.938691 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a744e75fc2e40909618999d05ec6d0f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:39.939359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a744e75fc2e40909618999d05ec6d0f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:39.955153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a744e75fc2e40909618999d05ec6d0f'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:39.968900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1344/2716] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:39 2022] DELETE /identity/v3/users/4a744e75fc2e40909618999d05ec6d0f => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:39.978576 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:39.979100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:39.979100 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:39.999728 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:40.001608 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:40.093867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1373/2717] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:40.103814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:40.105575 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:40.187721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de747899-3b25-4a8a-9f78-fca19fda9854 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:40.192804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:40.193873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:40.194368 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:40.194729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:40.208521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:40.210291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:40.228091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1345/2718] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:40.535084 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:40.536521 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:40.548768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b48a8-8713-4c34-b5ab-1f21ab1dab74 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:40.555161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:40.555811 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:40.556087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:40.556359 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] PATH_INFO: `/v3/projects/de56a7f061054cd68c1f676e8fd35d71` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:40.557500 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de56a7f061054cd68c1f676e8fd35d71)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:40.559443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:40.626730 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de56a7f061054cd68c1f676e8fd35d71', 'request_id': 'req-be8429f0-ff6c-4c4e-968c-898e78baf026'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.635336 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de56a7f061054cd68c1f676e8fd35d71', 'request_id': 'req-be8429f0-ff6c-4c4e-968c-898e78baf026'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.641771 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de56a7f061054cd68c1f676e8fd35d71', 'request_id': 'req-be8429f0-ff6c-4c4e-968c-898e78baf026'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.656493 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] The token cache is being invalidate because project de56a7f061054cd68c1f676e8fd35d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:40.656870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.657489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.663068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1374/2719] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:40 2022] DELETE /identity/v3/projects/de56a7f061054cd68c1f676e8fd35d71 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:40.664302 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:40.675005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7776e97e-cff6-4ce5-9ded-f9dd30df00cc service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:40.736005 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:40.736560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:40.736844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:40.737120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] PATH_INFO: `/v3/users/add32d3fd52b472fb32f4bb5b12687be` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:40.738291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=add32d3fd52b472fb32f4bb5b12687be)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:40.740210 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:40.787916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:40.791009 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'add32d3fd52b472fb32f4bb5b12687be'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.791825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'add32d3fd52b472fb32f4bb5b12687be'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.800341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'add32d3fd52b472fb32f4bb5b12687be'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.801133 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'add32d3fd52b472fb32f4bb5b12687be'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.815125 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'add32d3fd52b472fb32f4bb5b12687be'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:40.831538 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1346/2720] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:40 2022] DELETE /identity/v3/users/add32d3fd52b472fb32f4bb5b12687be => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:41.150018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:41.151520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:41.190509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8429f0-ff6c-4c4e-968c-898e78baf026 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:41.274827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:41.275656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:41.276016 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:41.276351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] PATH_INFO: `/v3/projects/dbf83ee83dc041e8a09dfd785c912a54` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:41.277534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbf83ee83dc041e8a09dfd785c912a54)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:41.279596 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:41.291438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:41.366575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbf83ee83dc041e8a09dfd785c912a54', 'request_id': 'req-14306bc2-b837-49e5-a589-9347fc939731'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.380052 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbf83ee83dc041e8a09dfd785c912a54', 'request_id': 'req-14306bc2-b837-49e5-a589-9347fc939731'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.388837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbf83ee83dc041e8a09dfd785c912a54', 'request_id': 'req-14306bc2-b837-49e5-a589-9347fc939731'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.423167 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] The token cache is being invalidate because project dbf83ee83dc041e8a09dfd785c912a54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:41.424272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.425215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.429836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1375/2721] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:41 2022] DELETE /identity/v3/projects/dbf83ee83dc041e8a09dfd785c912a54 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:41.431090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:41.432498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:41.452975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f98539-d4df-46f2-a59a-e39dc74d38d2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:41.543069 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:41.543646 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:41.543926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:41.544201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] PATH_INFO: `/v3/users/4350985e648e4da38d01fc251193e4a0` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:41.545388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] RBAC: Authorizing `identity:delete_user(user_id=4350985e648e4da38d01fc251193e4a0)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:41.547389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:41.635576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4350985e648e4da38d01fc251193e4a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.636268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4350985e648e4da38d01fc251193e4a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.653176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4350985e648e4da38d01fc251193e4a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.654021 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4350985e648e4da38d01fc251193e4a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.668112 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-2c81b640-f573-4936-98c6-db52506043de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4350985e648e4da38d01fc251193e4a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:41.683341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1347/2722] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:27:41 2022] DELETE /identity/v3/users/4350985e648e4da38d01fc251193e4a0 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:41.992749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:41.994229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:42.010739 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14306bc2-b837-49e5-a589-9347fc939731 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.084998 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.085610 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.085926 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.086447 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] PATH_INFO: `/v3/projects/269d0df4445047fb98f0870501b9d5a8` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.087609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] RBAC: Authorizing `identity:delete_project(project_id=269d0df4445047fb98f0870501b9d5a8)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.089564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.144251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '269d0df4445047fb98f0870501b9d5a8', 'request_id': 'req-7d8eed00-1778-4c65-be70-d500f0c8b81d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:42.151025 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '269d0df4445047fb98f0870501b9d5a8', 'request_id': 'req-7d8eed00-1778-4c65-be70-d500f0c8b81d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:42.157473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '269d0df4445047fb98f0870501b9d5a8', 'request_id': 'req-7d8eed00-1778-4c65-be70-d500f0c8b81d'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:42.171445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] The token cache is being invalidate because project 269d0df4445047fb98f0870501b9d5a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:27:42.171719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:42.174709 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-7d8eed00-1778-4c65-be70-d500f0c8b81d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:27:42.176591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1376/2723] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:27:41 2022] DELETE /identity/v3/projects/269d0df4445047fb98f0870501b9d5a8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:42.181261 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.181776 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.181881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.205625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:42.206888 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:42.240174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:42.269464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1348/2724] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:27:42.339581 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.339784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.340028 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.364275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:42.366135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:42.376794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1377/2725] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:42.381091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:42.392702 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9eec316-ece9-4724-a6bf-50639aa07eef None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.397925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.398780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.399059 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.399329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.400066 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:27:42.400378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.402183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.415387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1349/2726] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:27:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:27:42.419735 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:42.429721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2794ade-495f-4a9f-82b1-0a63ae7adabd None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.434949 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.435693 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.436011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.436298 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.436941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.439752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.474503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:42.486821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1378/2727] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:27:42 2022] POST /identity/v3/projects => generated 365 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:27:42.493361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:42.526794 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9854ea8-1062-4727-8345-7fb4a5c97873 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.649686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.650404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.650690 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.650964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.651664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.653627 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.702046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1350/2728] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:27:42 2022] POST /identity/v3/users => generated 283 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:27:42.707235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:42.720118 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345c341b-a1ef-4bc3-9ab7-4f66d6d4d579 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.735059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.736191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.736474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.736747 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.737440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.739538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.780802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1379/2729] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:27:42.786278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:42.797365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5918759-ec93-4d04-ba77-d89ea7107e51 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.807533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93447b74-5f72-4bf7-bd1d-e0ee9b6aef4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.808165 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447b74-5f72-4bf7-bd1d-e0ee9b6aef4c admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.808445 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447b74-5f72-4bf7-bd1d-e0ee9b6aef4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.808728 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93447b74-5f72-4bf7-bd1d-e0ee9b6aef4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.809376 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447b74-5f72-4bf7-bd1d-e0ee9b6aef4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.811248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93447b74-5f72-4bf7-bd1d-e0ee9b6aef4c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.823661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1351/2730] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:27:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:27:42.827947 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:42.839200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe46010-39f5-414d-85af-ce0fc1f14013 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:42.846901 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:42.847580 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.847859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.848132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] PATH_INFO: `/v3/projects/f5dcc8dd2d7f40c9866b08dcb4686dbd/users/b451caa425a942f58eadc0000caf5d4c/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.869486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5dcc8dd2d7f40c9866b08dcb4686dbd, user_id=b451caa425a942f58eadc0000caf5d4c, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:42.872404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:42.891522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1380/2731] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:27:42 2022] PUT /identity/v3/projects/f5dcc8dd2d7f40c9866b08dcb4686dbd/users/b451caa425a942f58eadc0000caf5d4c/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:27:42.901706 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790bad-6586-4b63-a863-fd4c5d696479 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:42.902082 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790bad-6586-4b63-a863-fd4c5d696479 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:42.902552 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6790bad-6586-4b63-a863-fd4c5d696479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:42.960324 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-a6790bad-6586-4b63-a863-fd4c5d696479 None None] MFA Rules not processed for user `b451caa425a942f58eadc0000caf5d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:42.966233 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-a6790bad-6586-4b63-a863-fd4c5d696479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:43.063864 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1352/2732] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:43.074818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:43.076792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:43.143852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8979898b-0eed-4a25-8827-1df6916c0a86 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:43.147733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:43.148480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:43.148836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:43.149181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:43.159262 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:43.160720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:43.170797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1381/2733] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:27:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:27:53.059283 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:53.166611 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:27:56.289521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:56.290042 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:56.290398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:56.310868 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:27:56.312623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:56.376777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1353/2734] 10.222.0.13 () {58 vars in 967 bytes} [Tue Oct 4 10:27:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:27:56.388936 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:27:56.391797 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:27:56.460592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382b4dc5-4854-4441-811a-c8e88f5b4268 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:27:56.465471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d988d97a-db91-46f9-a686-f28d9b7ca031 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:27:56.466061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988d97a-db91-46f9-a686-f28d9b7ca031 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:27:56.466369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988d97a-db91-46f9-a686-f28d9b7ca031 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:27:56.466736 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d988d97a-db91-46f9-a686-f28d9b7ca031 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:27:56.475008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988d97a-db91-46f9-a686-f28d9b7ca031 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:27:56.476583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d988d97a-db91-46f9-a686-f28d9b7ca031 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:27:56.484669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1382/2735] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:27:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:28:06.368987 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=25 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:06.485777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=32 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:17.320401 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:17.382405 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073b8098-de8a-42f5-8120-39a4ddba4493 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:17.386907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:17.387520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:17.387800 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:17.388067 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] PATH_INFO: `/v3/users/b451caa425a942f58eadc0000caf5d4c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:17.389186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b451caa425a942f58eadc0000caf5d4c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:17.391129 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:17.436031 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b451caa425a942f58eadc0000caf5d4c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.436667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b451caa425a942f58eadc0000caf5d4c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.443733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b451caa425a942f58eadc0000caf5d4c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.444356 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b451caa425a942f58eadc0000caf5d4c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.455911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b451caa425a942f58eadc0000caf5d4c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.469319 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1354/2736] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:28:17 2022] DELETE /identity/v3/users/b451caa425a942f58eadc0000caf5d4c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:28:17.473837 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:17.474341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:17.475326 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:17.494796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:17.495991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:17.553145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1383/2737] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:17.562798 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:17.564177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:17.620221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692efc95-203e-42e7-85ab-4599abeae497 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:17.623805 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cd8d3b-212d-4c24-bf6c-619767950440 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:17.624399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd8d3b-212d-4c24-bf6c-619767950440 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:17.624672 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd8d3b-212d-4c24-bf6c-619767950440 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:17.624955 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cd8d3b-212d-4c24-bf6c-619767950440 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:17.634404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd8d3b-212d-4c24-bf6c-619767950440 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:17.635818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cd8d3b-212d-4c24-bf6c-619767950440 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:17.645247 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1355/2738] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:17.910691 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:17.912341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:17.924321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe7e8c-90e4-4ae8-b9c0-564ac0458c6e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:17.929965 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:17.930638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:17.930918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:17.931192 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] PATH_INFO: `/v3/projects/f5dcc8dd2d7f40c9866b08dcb4686dbd` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:17.932468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5dcc8dd2d7f40c9866b08dcb4686dbd)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:17.934411 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:17.981712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5dcc8dd2d7f40c9866b08dcb4686dbd', 'request_id': 'req-6962421c-d062-4a6c-8138-cb1ce755c5cd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.987668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5dcc8dd2d7f40c9866b08dcb4686dbd', 'request_id': 'req-6962421c-d062-4a6c-8138-cb1ce755c5cd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:17.993418 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5dcc8dd2d7f40c9866b08dcb4686dbd', 'request_id': 'req-6962421c-d062-4a6c-8138-cb1ce755c5cd'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:18.006679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] The token cache is being invalidate because project f5dcc8dd2d7f40c9866b08dcb4686dbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:28:18.006962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:18.007590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-6962421c-d062-4a6c-8138-cb1ce755c5cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:18.009061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1384/2739] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:28:17 2022] DELETE /identity/v3/projects/f5dcc8dd2d7f40c9866b08dcb4686dbd => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:28:18.015678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.015964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.016240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.036523 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:18.037863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.054529 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:18.097561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1356/2740] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:18.105707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.106006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.106338 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.126435 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:18.127686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.136659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1385/2741] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:18.141012 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.151083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659adda3-1aa2-4645-8783-a9ffcad4c1cc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.155576 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.156167 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.156443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.156711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.157346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:28:18.157643 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.159248 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.170697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1357/2742] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:28:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:28:18.175059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.184352 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1580aa-c1e4-4a36-9c32-891297db5720 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.189127 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.189798 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.190075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.190403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.191051 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.194277 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.223294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1386/2743] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:28:18 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:28:18.227841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.237470 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fe3c10-ce66-414a-a102-9e415eba9fd0 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.291432 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.292026 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.292299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.292567 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.293189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.295142 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.316637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1358/2744] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:28:18 2022] POST /identity/v3/users => generated 287 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:28:18.321165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.331508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d239fbe-d604-4e66-b415-15a72219961e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.336251 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.336836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.337112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.337383 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.338020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.339659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.349751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1387/2745] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:28:18.353662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.363395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8274312-cdb3-4ab6-9e14-741916b3ca44 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.368689 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0e28c-2489-48ed-95d0-6ea7a659de38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.369285 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0e28c-2489-48ed-95d0-6ea7a659de38 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.369564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0e28c-2489-48ed-95d0-6ea7a659de38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.369839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0e28c-2489-48ed-95d0-6ea7a659de38 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.370506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0e28c-2489-48ed-95d0-6ea7a659de38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.372118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0e28c-2489-48ed-95d0-6ea7a659de38 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.383396 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1359/2746] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:28:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:28:18.387382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.397142 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67151875-9d63-42ca-8f1e-58bbad7a8b7c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.401676 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.402339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.402619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.402892 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] PATH_INFO: `/v3/projects/51313dd943014a499c923b64f8751e8c/users/7ff7d4964a90400e9139544688b3dee3/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.417722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] RBAC: Authorizing `identity:create_grant(project_id=51313dd943014a499c923b64f8751e8c, user_id=7ff7d4964a90400e9139544688b3dee3, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.420380 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.433002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1388/2747] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:28:18 2022] PUT /identity/v3/projects/51313dd943014a499c923b64f8751e8c/users/7ff7d4964a90400e9139544688b3dee3/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:28:18.443831 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e20f0-4af7-4999-89d1-0de11f3280ba None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.444118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e20f0-4af7-4999-89d1-0de11f3280ba None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.444389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e20f0-4af7-4999-89d1-0de11f3280ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.471624 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-d59e20f0-4af7-4999-89d1-0de11f3280ba None None] MFA Rules not processed for user `7ff7d4964a90400e9139544688b3dee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:18.472818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d59e20f0-4af7-4999-89d1-0de11f3280ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.536436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1360/2748] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:18.547902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:18.549688 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:18.572966 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:18.610781 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a663322d-a9c0-41e9-947d-22b41659616f admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:18.614405 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6dac91-fcd2-467f-9c64-b909053d2d52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:18.615682 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dac91-fcd2-467f-9c64-b909053d2d52 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:18.616401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dac91-fcd2-467f-9c64-b909053d2d52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:18.616671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6dac91-fcd2-467f-9c64-b909053d2d52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:18.625268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6dac91-fcd2-467f-9c64-b909053d2d52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:18.631339 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6dac91-fcd2-467f-9c64-b909053d2d52 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:18.636571 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1389/2749] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:18 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:19.992171 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:19.992626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:19.992731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:20.011767 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] MFA Rules not processed for user `7ff7d4964a90400e9139544688b3dee3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:20.012943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:20.019630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1361/2750] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:28.632979 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=17 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:30.007764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:34.873431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c6ef-5f37-4401-87f0-2c0afe871479 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:34.873911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c6ef-5f37-4401-87f0-2c0afe871479 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:34.874022 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8924c6ef-5f37-4401-87f0-2c0afe871479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:34.902089 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8924c6ef-5f37-4401-87f0-2c0afe871479 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:34.903299 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8924c6ef-5f37-4401-87f0-2c0afe871479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:34.982081 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1390/2751] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:28:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:35.003077 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:35.004497 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:35.104748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3e5a92-6cc6-4e40-b6e0-78f997163d17 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:35.109757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:35.110384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:35.110662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:35.110935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:35.118748 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:35.120436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:35.128260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1362/2752] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:28:35.634977 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b2004-d7b9-4474-b9c3-adf3c2dceebc None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:35.635267 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b2004-d7b9-4474-b9c3-adf3c2dceebc None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:35.636240 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8b2004-d7b9-4474-b9c3-adf3c2dceebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:35.655827 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-9b8b2004-d7b9-4474-b9c3-adf3c2dceebc None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:35.657031 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9b8b2004-d7b9-4474-b9c3-adf3c2dceebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:35.664161 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1391/2753] 10.222.0.13 () {58 vars in 967 bytes} [Tue Oct 4 10:28:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:35.676456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:35.687372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d24422-aec3-40ff-a144-9f8fdb092ec0 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:35.692335 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:35.692918 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:35.693193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:35.693464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:35.702126 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:35.703716 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:35.711625 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1363/2754] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:28:35.936982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8833-caf1-429d-b04b-1cebee7f5cc8 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:35.937424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8833-caf1-429d-b04b-1cebee7f5cc8 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:35.937508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ad8833-caf1-429d-b04b-1cebee7f5cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:35.957393 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-d1ad8833-caf1-429d-b04b-1cebee7f5cc8 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:35.958702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d1ad8833-caf1-429d-b04b-1cebee7f5cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:35.966233 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1392/2755] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:28:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:35.978662 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:35.989958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc3a37d-fc66-49b9-b23e-67a2dd82f313 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:35.995761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:35.996358 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:35.996640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:35.996919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:36.005114 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:36.006883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:36.015531 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1364/2756] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:28:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:28:36.352450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:36.352805 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:36.353158 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:36.383592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:36.384773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:36.392054 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1393/2757] 10.222.0.13 () {58 vars in 967 bytes} [Tue Oct 4 10:28:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:36.404937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:36.415422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced86247-d515-4501-912c-3da69a2a3979 service swift] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:36.421227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4059409-3e6a-47be-83d2-86b2f8b9be1b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:36.421817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059409-3e6a-47be-83d2-86b2f8b9be1b service swift] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:36.422102 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059409-3e6a-47be-83d2-86b2f8b9be1b service swift] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:36.422404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4059409-3e6a-47be-83d2-86b2f8b9be1b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:36.430301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4059409-3e6a-47be-83d2-86b2f8b9be1b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:36.431890 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4059409-3e6a-47be-83d2-86b2f8b9be1b service swift] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:36.439838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1365/2758] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:28:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:28:38.535663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:38.618900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3106d85-a079-4719-bd6f-dbe1aa44aef7 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:38.624575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:38.625214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:38.625488 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:38.625757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] PATH_INFO: `/v3/users/7ff7d4964a90400e9139544688b3dee3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:38.626911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ff7d4964a90400e9139544688b3dee3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:38.628840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:38.657225 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:38.676738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ff7d4964a90400e9139544688b3dee3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:38.677373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ff7d4964a90400e9139544688b3dee3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:38.684732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ff7d4964a90400e9139544688b3dee3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:38.685351 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ff7d4964a90400e9139544688b3dee3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:38.696904 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ff7d4964a90400e9139544688b3dee3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:38.709991 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1394/2759] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:28:38 2022] DELETE /identity/v3/users/7ff7d4964a90400e9139544688b3dee3 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:28:38.714764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:38.715270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:38.715270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:38.755438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:38.756647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:38.805251 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:38.814214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1366/2760] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:38.823937 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:38.825331 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:38.883673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd76eb9b-d765-4726-af47-2b40ace04955 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:38.887674 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3874b-d486-4c8d-83d5-06e99203dbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:38.888276 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3874b-d486-4c8d-83d5-06e99203dbbb service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:38.888553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3874b-d486-4c8d-83d5-06e99203dbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:38.888825 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3874b-d486-4c8d-83d5-06e99203dbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:38.898041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3874b-d486-4c8d-83d5-06e99203dbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:38.899540 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a3874b-d486-4c8d-83d5-06e99203dbbb service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:38.909480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1395/2761] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:39.079118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.080667 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:39.101622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9c2800-21f3-4c89-93b9-dd49fd03b4e1 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.109481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.110366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.110965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.111086 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] PATH_INFO: `/v3/projects/51313dd943014a499c923b64f8751e8c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.112546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51313dd943014a499c923b64f8751e8c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.114512 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.174083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51313dd943014a499c923b64f8751e8c', 'request_id': 'req-7a1599a7-ce23-45e7-828b-b04f6a0347f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:39.179532 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51313dd943014a499c923b64f8751e8c', 'request_id': 'req-7a1599a7-ce23-45e7-828b-b04f6a0347f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:39.187006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51313dd943014a499c923b64f8751e8c', 'request_id': 'req-7a1599a7-ce23-45e7-828b-b04f6a0347f2'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:39.198467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] The token cache is being invalidate because project 51313dd943014a499c923b64f8751e8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:28:39.198910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:39.199564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-7a1599a7-ce23-45e7-828b-b04f6a0347f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:28:39.201090 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1367/2762] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:28:39 2022] DELETE /identity/v3/projects/51313dd943014a499c923b64f8751e8c => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:28:39.205963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.206280 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.206578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.255371 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:39.256578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:39.318096 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1396/2763] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:39.326266 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.326502 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.326780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.347169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:39.348367 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:39.356682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1368/2764] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:39.360790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.370379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71fad0c-f94d-490f-9cc8-5b2633a4e0b6 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.374828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.375413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.375686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.375953 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.376590 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:28:39.376890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.378533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.390172 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1397/2765] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:28:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:28:39.394329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.403851 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6191-3fe7-4368-adfe-6fcef611a405 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.408650 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.409406 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.409684 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.409956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.410630 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.412537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.440145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1369/2766] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:28:39 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:28:39.444182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.454664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc101ad3-239d-4e16-b30e-cfcf95c5610b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.459197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:39.516229 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.516836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.517109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.517382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.518040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.520020 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.545765 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1398/2767] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:28:39 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:28:39.547018 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.557293 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59adf4b6-6913-4c1f-b58a-324ecf6b1b45 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.562061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.562692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.562975 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.563250 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.563891 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.565490 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.576495 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1370/2768] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:28:39.580349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.591738 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3f7e37-92c9-4d5d-946f-389dae71b249 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.598494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d401c53b-ef93-412b-89f0-247ca4bbf03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.599107 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401c53b-ef93-412b-89f0-247ca4bbf03f admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.599385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401c53b-ef93-412b-89f0-247ca4bbf03f admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.599655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d401c53b-ef93-412b-89f0-247ca4bbf03f admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.600302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401c53b-ef93-412b-89f0-247ca4bbf03f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.601920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d401c53b-ef93-412b-89f0-247ca4bbf03f admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.616655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1399/2769] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:28:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:28:39.620979 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.631655 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32362eb-46ce-40fd-9668-fb431298d0f5 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.638793 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.639460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.639739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.640015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] PATH_INFO: `/v3/projects/0b8a27bf66104e2c9abbdf03bfb6cbd6/users/95dca7fc896b46a6905e2f480a4e33c8/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.655583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8a27bf66104e2c9abbdf03bfb6cbd6, user_id=95dca7fc896b46a6905e2f480a4e33c8, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.658294 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.673467 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1371/2770] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:28:39 2022] PUT /identity/v3/projects/0b8a27bf66104e2c9abbdf03bfb6cbd6/users/95dca7fc896b46a6905e2f480a4e33c8/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:28:39.678303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1f0e-2d84-4f50-866d-eb40035fa58a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.678677 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1f0e-2d84-4f50-866d-eb40035fa58a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.679045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ff1f0e-2d84-4f50-866d-eb40035fa58a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.697877 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-41ff1f0e-2d84-4f50-866d-eb40035fa58a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:39.700027 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-41ff1f0e-2d84-4f50-866d-eb40035fa58a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:39.760323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1400/2771] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:39.770803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:39.772605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:39.831314 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851bce0e-2f0c-4c7d-9e18-7fde386fec49 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:39.834825 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e1c5f-5e50-48e3-a5c0-e59e90b7db01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:39.835422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1c5f-5e50-48e3-a5c0-e59e90b7db01 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:39.835694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1c5f-5e50-48e3-a5c0-e59e90b7db01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:39.835962 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e1c5f-5e50-48e3-a5c0-e59e90b7db01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:39.845485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e1c5f-5e50-48e3-a5c0-e59e90b7db01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:39.846937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e1c5f-5e50-48e3-a5c0-e59e90b7db01 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:39.856548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1372/2772] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:44.333442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.333442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.333946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.356522 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] MFA Rules not processed for user `c2fdbf946666403bad994a897335672c`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:44.358141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:44.470018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1401/2773] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:44.685422 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.685981 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.686148 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.721917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] MFA Rules not processed for user `95dca7fc896b46a6905e2f480a4e33c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:44.723213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:44.737017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:44.804015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1373/2774] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:44.809305 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:44.821784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4175acb7-e00d-4a35-ab3f-ce08337b8a2c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:44.827842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:44.828549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.828834 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.829110 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.829753 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:44.831922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:44.867495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1402/2775] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:28:44 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:28:44.868596 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:44.880256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d7a43a-7c79-49a3-8373-782c4cbb2efc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:44.945804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:44.947048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.947385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.947722 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:44.948465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:44.950925 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:44.978253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1374/2776] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:28:44 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:28:44.981094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:44.993176 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9dd18b-abe4-4371-bab8-e893f0d028e1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:44.998725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:44.999372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:44.999658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:44.999982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:45.000627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:45.002270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:45.009049 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:45.019208 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1403/2777] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:28:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:28:45.023032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:45.033374 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f0e7c-148d-4336-9ee5-39ee075046c3 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:45.038061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:45.039348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:45.039348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:45.039348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] PATH_INFO: `/v3/projects/59a7ed0a5b2142169a34832446be67ea/users/5895e009e8b746ff842932bc45d6f6f7/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:45.055469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a7ed0a5b2142169a34832446be67ea, user_id=5895e009e8b746ff842932bc45d6f6f7, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:45.058783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:45.073475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1375/2778] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:28:45 2022] PUT /identity/v3/projects/59a7ed0a5b2142169a34832446be67ea/users/5895e009e8b746ff842932bc45d6f6f7/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:28:45.077414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:45.087909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06282a7-0b4d-4fa5-b929-17e67c7b40ec admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:45.161426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168f51b-0cc1-4398-a9f9-231515327cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:45.162141 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f51b-0cc1-4398-a9f9-231515327cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:45.162636 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f51b-0cc1-4398-a9f9-231515327cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:45.163069 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168f51b-0cc1-4398-a9f9-231515327cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:45.163856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f51b-0cc1-4398-a9f9-231515327cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:45.165777 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168f51b-0cc1-4398-a9f9-231515327cd7 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:45.176030 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1404/2779] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:28:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:28:45.180272 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:45.192065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05988e8-5157-42c9-93c2-7c95a7138e60 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:45.197256 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:45.197929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:45.198339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:45.199219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] PATH_INFO: `/v3/projects/59a7ed0a5b2142169a34832446be67ea/users/5895e009e8b746ff842932bc45d6f6f7/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:45.201527 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59a7ed0a5b2142169a34832446be67ea, user_id=5895e009e8b746ff842932bc45d6f6f7, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:45.204178 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:45.214905 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1376/2780] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:28:45 2022] PUT /identity/v3/projects/59a7ed0a5b2142169a34832446be67ea/users/5895e009e8b746ff842932bc45d6f6f7/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:28:50.755288 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430e350-827a-40a5-9f32-13a627868649 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:50.755773 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430e350-827a-40a5-9f32-13a627868649 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:50.756098 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4430e350-827a-40a5-9f32-13a627868649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:50.787826 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-4430e350-827a-40a5-9f32-13a627868649 None None] MFA Rules not processed for user `5895e009e8b746ff842932bc45d6f6f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:50.789053 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-4430e350-827a-40a5-9f32-13a627868649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:50.862675 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1405/2781] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:28:50.876737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:28:50.878737 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:50.953333 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:28:50.988454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551e0b4b-b331-4f3f-a2a5-ca426a39a94e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:28:50.994325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:28:50.994941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:50.995392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:50.995749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:51.062773 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:28:51.064595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:28:51.075174 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1377/2782] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:28:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:28:52.189437 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:28:52.189927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:28:52.190041 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:28:52.209655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] MFA Rules not processed for user `95dca7fc896b46a6905e2f480a4e33c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:28:52.210887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:28:52.218909 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1406/2783] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:28:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:01.070299 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=36 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:02.204733 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=66 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:08.684902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:08.686353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:08.777917 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433eef17-c8cd-428c-aff6-20460b1e17cf service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:08.782949 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:08.783541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:08.783819 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:08.784089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:08.785780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:08.796993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:08.798572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d993623a-9c8c-4a86-ac64-7af1a93fd7b8 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:08.808230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1378/2784] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:29:18.803978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=41 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:24.888619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:24.890112 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:24.965609 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069c7f44-4f4f-4286-b24e-20ba725bb41a None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:24.969369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:24.969963 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:24.970266 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:24.970534 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:24.979214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:24.980630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:24.990817 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1407/2785] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:29:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:29:34.986589 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=35 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:29:59.285537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a003fe4-2b22-446d-ad8b-2e379e830916 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:59.286049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a003fe4-2b22-446d-ad8b-2e379e830916 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:59.286158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a003fe4-2b22-446d-ad8b-2e379e830916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:59.306516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1a003fe4-2b22-446d-ad8b-2e379e830916 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:29:59.307744 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1a003fe4-2b22-446d-ad8b-2e379e830916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:59.364177 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1379/2786] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:29:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:29:59.413971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:29:59.415460 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:29:59.497450 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87f54f-3815-43fe-b1bb-94961e70ca20 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:29:59.503599 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:29:59.504964 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:29:59.505242 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:29:59.505507 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:29:59.514129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:29:59.516385 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:29:59.524658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1408/2787] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:29:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:30:09.356900 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=25 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:09.521455 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=32 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:16.431147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89965e4-bc3c-4f90-8ea8-c3837e3a54c9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:16.431648 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89965e4-bc3c-4f90-8ea8-c3837e3a54c9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:16.431751 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89965e4-bc3c-4f90-8ea8-c3837e3a54c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:16.451070 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c89965e4-bc3c-4f90-8ea8-c3837e3a54c9 None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:16.452260 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c89965e4-bc3c-4f90-8ea8-c3837e3a54c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:16.459676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1380/2788] 10.222.0.13 () {58 vars in 1009 bytes} [Tue Oct 4 10:30:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:16.472109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:16.482524 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b87a37-f9e4-4c64-8f60-4d8d5e9e2d21 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:16.487927 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:16.488501 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:16.488774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:16.489047 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:16.496944 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:16.498530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:16.507188 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1409/2789] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:30:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 04 10:30:26.446297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:26.503760 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:30.822469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:30.822920 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:30.823045 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:30.846577 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] MFA Rules not processed for user `9f385f2a341a4162af72414f4dc0ee91`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:30:30.847763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:30:30.855152 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1381/2790] 10.222.0.13 () {58 vars in 967 bytes} [Tue Oct 4 10:30:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:30:30.867829 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:30:30.878448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312460f3-4ba4-4e7d-962e-d0b11217fa98 service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:30:30.883397 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf8cf97352f244d4b87c5927ee451dd9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:30:30.883925 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:30:30.884197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:30:30.884494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:30:30.893001 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:30:30.894597 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:30:30.902631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1410/2791] 10.222.0.13 () {60 vars in 1384 bytes} [Tue Oct 4 10:30:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 04 10:30:40.840724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:30:40.899268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:05.348788 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:05.350230 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:05.425821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66c7839-194a-4fd1-b6e5-adf6ae73abcc None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:05.429379 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:05.429968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:05.430262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:05.430533 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:05.432213 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:05.490513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:05.491937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:05.500711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1382/2792] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:05 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:05.847316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:05.856879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c140a42b-212f-4458-abd7-8ef9297cc38f service swift] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:05.860463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:05.861046 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:05.861314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:05.861583 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:05.863353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:05.876867 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:05.878300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:05.888950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1411/2793] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:07.074227 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:07.084275 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e49c978-045b-4b21-a8a2-0ce97c0433a3 service cinder] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:07.087841 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:07.088418 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:07.088692 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:07.088958 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:07.090675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:07.104832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:07.106253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:07.116818 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1383/2794] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:08.321409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:08.322824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:08.338627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6b07b1-138a-4e1f-9701-2be90e1387e1 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:08.345722 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:08.355950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:08.357183 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:08.358182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:08.363912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:08.386577 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:08.388551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:08.413265 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1412/2795] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:12.633134 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:12.642574 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b4ec07-765d-4acf-8891-2dc690d2d7bc service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:12.646277 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:12.646823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:12.647103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:12.647378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:12.649122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:12.711510 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:12.712921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:12.722587 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1384/2796] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:15.911037 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:15.923133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e14620-386f-4a0a-98ac-1779aeb8ba64 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:15.928119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:15.928712 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:15.928984 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:15.929253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:15.930955 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:15.942423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:15.944002 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:15.954686 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1413/2797] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:15.971659 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:15.973384 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:15.986267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6133df79-a73e-4dd5-9aa1-8b0f380871b5 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:15.992058 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:15.992682 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:15.992957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:15.993226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] PATH_INFO: `/v3/users/c2fdbf946666403bad994a897335672c` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:15.994378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2fdbf946666403bad994a897335672c)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:15.996270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.008361 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=100 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:16.042408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2fdbf946666403bad994a897335672c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.043052 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2fdbf946666403bad994a897335672c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.050402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2fdbf946666403bad994a897335672c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.051020 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2fdbf946666403bad994a897335672c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.062996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-ae4d9817-2971-40b0-8631-566ca5f6de80 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2fdbf946666403bad994a897335672c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.075752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1385/2798] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:31:15 2022] DELETE /identity/v3/users/c2fdbf946666403bad994a897335672c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:31:16.249824 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.262175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef836b1-975b-4523-9eac-2177ea67251b service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.319707 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.320329 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.320618 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.320896 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] PATH_INFO: `/v3/projects/25a4307806824ec4bd3683b8ca7b9752` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.321993 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25a4307806824ec4bd3683b8ca7b9752)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.323905 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.374150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25a4307806824ec4bd3683b8ca7b9752', 'request_id': 'req-0e19737f-6347-4a94-8e10-9b4ff7614890'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.380616 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25a4307806824ec4bd3683b8ca7b9752', 'request_id': 'req-0e19737f-6347-4a94-8e10-9b4ff7614890'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.386271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25a4307806824ec4bd3683b8ca7b9752', 'request_id': 'req-0e19737f-6347-4a94-8e10-9b4ff7614890'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.399357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] The token cache is being invalidate because project 25a4307806824ec4bd3683b8ca7b9752 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:31:16.399644 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.400273 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-0e19737f-6347-4a94-8e10-9b4ff7614890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:31:16.401763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1414/2799] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:31:16 2022] DELETE /identity/v3/projects/25a4307806824ec4bd3683b8ca7b9752 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:31:16.406927 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.407799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.408301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.427321 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:16.428569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:16.488141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1386/2800] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:16.496019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.496300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.496566 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.515990 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:16.517859 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:16.525632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1415/2801] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:31:16.529858 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.540231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64821930-8506-46b3-aec9-29c0586e2dfb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.544723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.545310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.545584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.545853 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.546525 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:31:16.546822 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.548479 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.559694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1387/2802] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:31:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:31:16.563980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.572956 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c42953-e3a7-4566-b04a-55a8c9b10d87 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.577480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.578063 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.578373 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.578670 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.579807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.581680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.622508 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1416/2803] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:31:16 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:31:16.629107 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.636772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872cd1a7-c642-40da-a7c8-56c51c585bbb admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.693226 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.693965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.694284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.694563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.695196 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.697156 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.719310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1388/2804] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:31:16 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:31:16.724181 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.734357 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896b512-c6ce-4341-a721-05c531a75c12 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.740882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.741503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.741780 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.742050 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.742790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.744414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.754662 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1417/2805] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:31:16.758504 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.764679 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:16.768377 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b185bf03-8119-4d7b-8ebf-63230fb70723 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.774140 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38bc3bf-a03b-4693-84c1-9ed593518902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.774763 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bc3bf-a03b-4693-84c1-9ed593518902 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.775276 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bc3bf-a03b-4693-84c1-9ed593518902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.775723 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38bc3bf-a03b-4693-84c1-9ed593518902 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.776499 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bc3bf-a03b-4693-84c1-9ed593518902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.778253 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38bc3bf-a03b-4693-84c1-9ed593518902 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.789085 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1389/2806] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:31:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:31:16.793189 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.798917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:16.805487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4350b338-a70e-45a9-aff5-eae58ecd2c89 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.810740 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.811379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.811659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.811962 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] PATH_INFO: `/v3/projects/a19b1d07ac66407088b75b924dba9930/users/38fc6dec2b8e4ee2a16260516489d441/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.826372 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19b1d07ac66407088b75b924dba9930, user_id=38fc6dec2b8e4ee2a16260516489d441, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:16.829065 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:16.841768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1418/2807] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:31:16 2022] PUT /identity/v3/projects/a19b1d07ac66407088b75b924dba9930/users/38fc6dec2b8e4ee2a16260516489d441/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:31:16.847592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84d114-3b41-407b-a345-adbdc5b180f3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.847878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84d114-3b41-407b-a345-adbdc5b180f3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.848153 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84d114-3b41-407b-a345-adbdc5b180f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:16.867239 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1d84d114-3b41-407b-a345-adbdc5b180f3 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:16.868454 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1d84d114-3b41-407b-a345-adbdc5b180f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:16.926062 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1390/2808] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:31:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:16.935451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:16.937419 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:16.993983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2eaa60-55e0-46f3-9e2e-b0e24f72eb07 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:16.997582 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:16.998177 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:16.998481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:16.998755 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:17.007721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:17.009135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:17.018514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1419/2809] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:20.660790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42d9cb-b211-43c7-ad75-278239c214ca None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:20.661332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42d9cb-b211-43c7-ad75-278239c214ca None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:20.661430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc42d9cb-b211-43c7-ad75-278239c214ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:20.689398 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-bc42d9cb-b211-43c7-ad75-278239c214ca None None] MFA Rules not processed for user `38fc6dec2b8e4ee2a16260516489d441`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:20.690584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-bc42d9cb-b211-43c7-ad75-278239c214ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:20.764169 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1391/2810] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:31:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:20.779490 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:20.781442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:20.858871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b08b3e0-a3bb-4653-a347-498b98073d9a service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:20.864453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:20.865076 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:20.865360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:20.865634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:20.873685 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:20.875337 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:20.884379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1420/2811] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:22.249145 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:22.249647 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:22.249772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:22.269664 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] MFA Rules not processed for user `38fc6dec2b8e4ee2a16260516489d441`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:31:22.270907 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:22.277889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1392/2812] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:31:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:31:22.914087 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:22.915794 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:22.997871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3844a3-8003-472b-898d-88f6122d2d89 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:23.003157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:23.003886 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:23.004170 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:23.004438 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:23.006147 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:23.015876 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:23.019211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:23.021303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:23.033422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1421/2813] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:25.367215 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:25.369023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:25.392307 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dae6049-98c7-46cb-8d08-2e5d9c4f684c None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:25.396838 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:25.397715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:25.398056 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:25.398447 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:25.401010 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:25.411709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:25.413122 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:25.421485 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1393/2814] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:31.691514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:31.692821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:31.718613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b712a-13c8-418c-8fe5-1cbda02985d8 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:31.728848 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:31.729564 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:31.729942 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:31.730415 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:31.732679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:31.747620 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:31.749333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:31.765778 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1422/2815] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:35.305199 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:35.319051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b401254-ae38-4a72-88c1-482923c9280b service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:35.325709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:35.326414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:35.326869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:35.327446 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:35.331209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:35.345332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:35.348138 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8ff3b6-12ad-4707-974e-49b16fc306fc service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:35.360397 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1394/2816] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:37.853672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:31:37.863384 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df72e8-6f4c-4c1f-9d93-1b90ba8c4212 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:31:37.866912 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:31:37.867496 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:31:37.867768 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:31:37.868038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:31:37.869752 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:31:37.879243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:31:37.880656 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:31:37.888530 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1423/2817] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:31:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:31:45.355616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=77 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:31:47.884950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=14 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:32:21.790546 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:32:21.791096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:32:21.791096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:32:21.814339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] MFA Rules not processed for user `38fc6dec2b8e4ee2a16260516489d441`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:32:21.815521 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:32:21.822320 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1395/2818] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:32:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:32:31.805354 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:13.246456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:13.248587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:13.264552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bfba8d-57f0-4d0c-8a2a-e724fef8252f service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:13.269623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:13.270239 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:13.270506 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:13.270774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:13.272480 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:13.284145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:13.285804 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:13.299232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1424/2819] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:33:17.682626 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:17.684287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:17.701778 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5a9ad6-c0f3-4b6f-92c6-3b1f4db7ae20 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:17.706632 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:17.707298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:17.707579 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:17.707845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] PATH_INFO: `/v3/users/95dca7fc896b46a6905e2f480a4e33c8` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:17.708964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95dca7fc896b46a6905e2f480a4e33c8)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:17.710961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:17.760113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95dca7fc896b46a6905e2f480a4e33c8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:17.761132 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95dca7fc896b46a6905e2f480a4e33c8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:17.769944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95dca7fc896b46a6905e2f480a4e33c8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:17.770752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95dca7fc896b46a6905e2f480a4e33c8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:17.783830 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95dca7fc896b46a6905e2f480a4e33c8'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:17.798605 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1396/2820] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:33:17 2022] DELETE /identity/v3/users/95dca7fc896b46a6905e2f480a4e33c8 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:33:17.973121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:17.984810 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5402ce9f-b2c6-4bed-a35e-7bc71399c9b6 service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:18.042871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:18.043498 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:18.043772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:18.044044 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] PATH_INFO: `/v3/projects/0b8a27bf66104e2c9abbdf03bfb6cbd6` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:18.045194 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8a27bf66104e2c9abbdf03bfb6cbd6)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:18.047132 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:18.093721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8a27bf66104e2c9abbdf03bfb6cbd6', 'request_id': 'req-ba13c62d-379f-409a-8129-0eff5117dad9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.101334 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8a27bf66104e2c9abbdf03bfb6cbd6', 'request_id': 'req-ba13c62d-379f-409a-8129-0eff5117dad9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.107309 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8a27bf66104e2c9abbdf03bfb6cbd6', 'request_id': 'req-ba13c62d-379f-409a-8129-0eff5117dad9'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.119428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] The token cache is being invalidate because project 0b8a27bf66104e2c9abbdf03bfb6cbd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:33:18.119723 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.120349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.121869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1425/2821] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:33:17 2022] DELETE /identity/v3/projects/0b8a27bf66104e2c9abbdf03bfb6cbd6 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:33:18.126176 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:18.127595 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:18.140884 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54102ba4-23f1-4710-8cb5-21a18d77a748 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:18.196926 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:18.197543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:18.197821 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:18.198091 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] PATH_INFO: `/v3/users/5895e009e8b746ff842932bc45d6f6f7` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:18.199236 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5895e009e8b746ff842932bc45d6f6f7)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:18.201729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:18.245404 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5895e009e8b746ff842932bc45d6f6f7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.246063 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5895e009e8b746ff842932bc45d6f6f7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.253708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5895e009e8b746ff842932bc45d6f6f7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.254392 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5895e009e8b746ff842932bc45d6f6f7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.266583 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-6efdfed7-a4df-4f8d-8f4c-1e26bf395362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5895e009e8b746ff842932bc45d6f6f7'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.268173 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:18.281661 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1397/2822] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:33:18 2022] DELETE /identity/v3/users/5895e009e8b746ff842932bc45d6f6f7 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:33:18.457468 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:18.459212 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:18.475980 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba13c62d-379f-409a-8129-0eff5117dad9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:18.534349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:18.534972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:18.535250 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:18.535517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] PATH_INFO: `/v3/projects/59a7ed0a5b2142169a34832446be67ea` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:18.536614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=59a7ed0a5b2142169a34832446be67ea)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:18.538547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:18.559223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:18.585535 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59a7ed0a5b2142169a34832446be67ea', 'request_id': 'req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.591543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59a7ed0a5b2142169a34832446be67ea', 'request_id': 'req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.598145 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59a7ed0a5b2142169a34832446be67ea', 'request_id': 'req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.610751 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] The token cache is being invalidate because project 59a7ed0a5b2142169a34832446be67ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:33:18.611039 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.611648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:33:18.613106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1426/2823] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:33:18 2022] DELETE /identity/v3/projects/59a7ed0a5b2142169a34832446be67ea => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:33:26.614402 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:26.614916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:26.615017 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:26.634296 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] MFA Rules not processed for user `38fc6dec2b8e4ee2a16260516489d441`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:33:26.635481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:26.689686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1398/2824] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:33:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:33:27.255481 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:33:27.256852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:27.313643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f91fcf-6b69-41d1-b0cb-5cce8eb1e7ae admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:33:27.317144 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:33:27.317783 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:33:27.318003 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:33:27.318300 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:33:27.319958 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:33:27.329727 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:33:27.331830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb71c19-7122-4d66-8bb7-7be02910c91e service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:33:27.341263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1427/2825] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:33:36.681833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=27 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:33:37.337406 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:34:29.344482 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:34:29.346863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:34:29.422956 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044e0c2-ddcf-4bc1-80dd-9188eeecbce9 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:34:29.427908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:34:29.428520 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:34:29.428786 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:34:29.429057 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:34:29.430764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:34:29.442547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:34:29.444051 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:34:29.454188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1399/2826] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:34:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:34:32.825426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:34:32.826013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:34:32.826013 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:34:32.845850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] MFA Rules not processed for user `38fc6dec2b8e4ee2a16260516489d441`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:34:32.847328 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:34:32.854111 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1428/2827] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:34:32 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:34:39.449550 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=32 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:34:42.840443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:43.005559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.007093 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:43.084019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1bac4f-b528-4ea6-b4da-d7aa731b87c6 service placement] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.088547 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.089163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.089443 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.089717 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] PATH_INFO: `/v3/users/38fc6dec2b8e4ee2a16260516489d441` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.090847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] RBAC: Authorizing `identity:delete_user(user_id=38fc6dec2b8e4ee2a16260516489d441)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.092896 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.143967 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38fc6dec2b8e4ee2a16260516489d441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.144675 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38fc6dec2b8e4ee2a16260516489d441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.152541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38fc6dec2b8e4ee2a16260516489d441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.153187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38fc6dec2b8e4ee2a16260516489d441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.168847 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-45ca657d-169b-4012-ba83-9dbb598b1fcc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38fc6dec2b8e4ee2a16260516489d441'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.182915 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1400/2828] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:43 2022] DELETE /identity/v3/users/38fc6dec2b8e4ee2a16260516489d441 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:43.354743 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.373486 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e1ad95-0696-4e99-a6c3-0ac419be889e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.431601 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.432263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.432626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.432968 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] PATH_INFO: `/v3/projects/a19b1d07ac66407088b75b924dba9930` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.434094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19b1d07ac66407088b75b924dba9930)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.436146 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.486792 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19b1d07ac66407088b75b924dba9930', 'request_id': 'req-cee52844-2322-49bf-8157-03ca018954db'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.492913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19b1d07ac66407088b75b924dba9930', 'request_id': 'req-cee52844-2322-49bf-8157-03ca018954db'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.498548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19b1d07ac66407088b75b924dba9930', 'request_id': 'req-cee52844-2322-49bf-8157-03ca018954db'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.512623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] The token cache is being invalidate because project a19b1d07ac66407088b75b924dba9930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81361) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:43.512903 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.513526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-cee52844-2322-49bf-8157-03ca018954db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:43.516363 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1429/2829] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:43 2022] DELETE /identity/v3/projects/a19b1d07ac66407088b75b924dba9930 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:43.521074 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.521378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.521746 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.543168 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:43.544468 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:43.610147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1401/2830] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:43.621699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.621987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.622315 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.643703 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:43.644897 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:43.652659 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1430/2831] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:43.656872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.666556 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d9fcc6-1577-452e-a3dc-ee90c6c3817a None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.671270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.671869 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.672143 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.672414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] PATH_INFO: `/v3/domains` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.673049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81362) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:43.673346 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.674963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.686749 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1402/2832] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:43.690871 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.699946 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41cc3ad-5d29-4f0a-9f9a-b2212f8057ae None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.705333 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.705922 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.706197 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.706494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.707121 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.709005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.735367 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1431/2833] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:43 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:43.739824 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.750436 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78da9931-dd18-43a4-bdc4-905bc0bb4341 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.754150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:43.809061 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.809658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.809938 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.810234 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.811496 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.813388 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.833947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1403/2834] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:43 2022] POST /identity/v3/users => generated 285 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:43.838557 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.849664 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b719619-b582-4626-a8d4-5cf5f989b169 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.854408 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.855006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.855302 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.855575 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.856215 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.858542 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.868422 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1432/2835] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:43.872310 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.882228 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceffac8f-886b-4a69-bcc1-5e6c87c6a9b7 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.886768 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff37efc8-0ec8-4641-833d-071a3cb48f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.887385 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37efc8-0ec8-4641-833d-071a3cb48f26 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.887665 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37efc8-0ec8-4641-833d-071a3cb48f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.887940 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff37efc8-0ec8-4641-833d-071a3cb48f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.888573 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37efc8-0ec8-4641-833d-071a3cb48f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.890149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff37efc8-0ec8-4641-833d-071a3cb48f26 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.900910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1404/2836] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:43.905671 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:43.915456 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876fe89c-b763-477d-bc5e-8844e407fde0 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:43.919985 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:43.921303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.921591 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.921862 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] PATH_INFO: `/v3/projects/7ca6ea4e4f3b4191bfcf38f7ba40f9a5/users/fb323e434d3e4ad2a06c8ce60aced9c3/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.936719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca6ea4e4f3b4191bfcf38f7ba40f9a5, user_id=fb323e434d3e4ad2a06c8ce60aced9c3, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:43.939440 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:43.952223 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1433/2837] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:43 2022] PUT /identity/v3/projects/7ca6ea4e4f3b4191bfcf38f7ba40f9a5/users/fb323e434d3e4ad2a06c8ce60aced9c3/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:43.963184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:43.963486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:43.963753 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:43.991772 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] MFA Rules not processed for user `fb323e434d3e4ad2a06c8ce60aced9c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:43.992995 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:44.060620 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1405/2838] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:44.067166 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.099994 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:44.133857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef0d91-d9e5-43e7-95d3-f93f3af99ce9 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.138630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.140322 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.140603 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.140899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.141553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.143495 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.170059 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1434/2839] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:44 2022] POST /identity/v3/projects => generated 367 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:44.174850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.185378 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b30b0b0-ba69-4cd9-9339-135b648319d3 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.240410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.241008 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.241287 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.241560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] PATH_INFO: `/v3/users` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.242194 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.244109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.265089 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1406/2840] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:44 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:44.269175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.280186 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e752f-2a24-4458-a6d1-bd628c0575bb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.284769 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.285381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.285655 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.285919 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.286574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.288163 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.298796 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1435/2841] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:44.302807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.308816 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:44.312781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f7428d-0069-4ddb-be44-86c0fd24b3ab admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.318506 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.319150 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.319425 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.319697 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] PATH_INFO: `/v3/projects/d3c7502806e04b75831f8d5934865fff/users/057d712b0cec4d7491b264b32fa7e3f0/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.335423 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c7502806e04b75831f8d5934865fff, user_id=057d712b0cec4d7491b264b32fa7e3f0, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.338268 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.353610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1407/2842] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:44 2022] PUT /identity/v3/projects/d3c7502806e04b75831f8d5934865fff/users/057d712b0cec4d7491b264b32fa7e3f0/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:44.357673 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.367623 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8420c1-227a-49a0-b3c7-7e211e01bbee admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.424981 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76e6232-845a-4de5-bc14-dd763e489396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.425509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6232-845a-4de5-bc14-dd763e489396 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.425789 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6232-845a-4de5-bc14-dd763e489396 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.426067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76e6232-845a-4de5-bc14-dd763e489396 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.426729 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e6232-845a-4de5-bc14-dd763e489396 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.428321 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76e6232-845a-4de5-bc14-dd763e489396 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.439412 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1436/2843] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:44 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 04 10:35:44.443770 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.454558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bdd13d-8982-44eb-a0a4-1bbdac4689b8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.459119 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.459783 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.460065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.460341 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] PATH_INFO: `/v3/projects/d3c7502806e04b75831f8d5934865fff/users/057d712b0cec4d7491b264b32fa7e3f0/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.462244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c7502806e04b75831f8d5934865fff, user_id=057d712b0cec4d7491b264b32fa7e3f0, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.464857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.475638 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1408/2844] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:44 2022] PUT /identity/v3/projects/d3c7502806e04b75831f8d5934865fff/users/057d712b0cec4d7491b264b32fa7e3f0/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:44.485514 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.485807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.486109 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.516316 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] MFA Rules not processed for user `057d712b0cec4d7491b264b32fa7e3f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:44.517578 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:44.584561 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1437/2845] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:44 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:44.595117 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:44.596790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:44.654636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08a9b29-0e77-44bc-a394-012ae89e5b36 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:44.658279 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860148f7-72fa-407e-8f8f-584410df4c00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:44.658873 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860148f7-72fa-407e-8f8f-584410df4c00 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:44.659149 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860148f7-72fa-407e-8f8f-584410df4c00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:44.659421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860148f7-72fa-407e-8f8f-584410df4c00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:44.668180 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860148f7-72fa-407e-8f8f-584410df4c00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:44.669588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860148f7-72fa-407e-8f8f-584410df4c00 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:44.679326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1409/2846] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:46.022377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.046830 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:46.085259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb95648b-3f5f-45e8-ae96-d5cdc759789c None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.089697 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.090340 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.090615 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.090882 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] PATH_INFO: `/v3/users/fb323e434d3e4ad2a06c8ce60aced9c3` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.092067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb323e434d3e4ad2a06c8ce60aced9c3)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.094071 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.138083 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb323e434d3e4ad2a06c8ce60aced9c3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.139409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb323e434d3e4ad2a06c8ce60aced9c3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.146838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb323e434d3e4ad2a06c8ce60aced9c3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.147458 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb323e434d3e4ad2a06c8ce60aced9c3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.159095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb323e434d3e4ad2a06c8ce60aced9c3'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.171106 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1438/2847] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:46 2022] DELETE /identity/v3/users/fb323e434d3e4ad2a06c8ce60aced9c3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:46.175584 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.176278 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.176637 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.195686 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:46.197055 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:46.255147 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1410/2848] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:46.264996 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.266445 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:46.322245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae691ec1-fc82-405e-a9fa-55c9c4bad503 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.325776 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.326494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.326900 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.327173 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.336019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.337430 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.348382 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1439/2849] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:46.621006 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.622478 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:46.641189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8d5102-a14d-4391-815c-a11be2316f59 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.646187 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.646974 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.647444 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.647791 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] PATH_INFO: `/v3/projects/7ca6ea4e4f3b4191bfcf38f7ba40f9a5` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.649483 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca6ea4e4f3b4191bfcf38f7ba40f9a5)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.651494 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.699939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca6ea4e4f3b4191bfcf38f7ba40f9a5', 'request_id': 'req-377d9496-99c7-4ba4-b1e3-7a014aff8f41'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.705375 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca6ea4e4f3b4191bfcf38f7ba40f9a5', 'request_id': 'req-377d9496-99c7-4ba4-b1e3-7a014aff8f41'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.712469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca6ea4e4f3b4191bfcf38f7ba40f9a5', 'request_id': 'req-377d9496-99c7-4ba4-b1e3-7a014aff8f41'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.713756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:46.726548 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] The token cache is being invalidate because project 7ca6ea4e4f3b4191bfcf38f7ba40f9a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:46.726833 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.727460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.728934 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1411/2850] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:46 2022] DELETE /identity/v3/projects/7ca6ea4e4f3b4191bfcf38f7ba40f9a5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:46.733899 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:46.743679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5642b610-c082-451f-b428-5a8f8442c157 service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:46.784982 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:46.801117 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:46.801737 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:46.802017 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:46.802323 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] PATH_INFO: `/v3/users/057d712b0cec4d7491b264b32fa7e3f0` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:46.803428 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] RBAC: Authorizing `identity:delete_user(user_id=057d712b0cec4d7491b264b32fa7e3f0)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:46.805344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:46.851217 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '057d712b0cec4d7491b264b32fa7e3f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.851852 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '057d712b0cec4d7491b264b32fa7e3f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.858918 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '057d712b0cec4d7491b264b32fa7e3f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.859532 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '057d712b0cec4d7491b264b32fa7e3f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.871006 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-dcc09238-f60d-44de-9f97-c7c86b2c2910 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '057d712b0cec4d7491b264b32fa7e3f0'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:46.883821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1440/2851] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:35:46 2022] DELETE /identity/v3/users/057d712b0cec4d7491b264b32fa7e3f0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 04 10:35:47.162465 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.163881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.177588 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d9496-99c7-4ba4-b1e3-7a014aff8f41 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.235414 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.236049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.236325 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.236599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] PATH_INFO: `/v3/projects/d3c7502806e04b75831f8d5934865fff` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.237708 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c7502806e04b75831f8d5934865fff)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.239966 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.285163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c7502806e04b75831f8d5934865fff', 'request_id': 'req-e527df97-5ca1-4354-8109-43f4aa77c7a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:47.290640 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c7502806e04b75831f8d5934865fff', 'request_id': 'req-e527df97-5ca1-4354-8109-43f4aa77c7a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:47.298158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c7502806e04b75831f8d5934865fff', 'request_id': 'req-e527df97-5ca1-4354-8109-43f4aa77c7a0'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:47.308902 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] The token cache is being invalidate because project d3c7502806e04b75831f8d5934865fff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:35:47.309183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:47.309804 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-e527df97-5ca1-4354-8109-43f4aa77c7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:35:47.311911 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1412/2852] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:35:47 2022] DELETE /identity/v3/projects/d3c7502806e04b75831f8d5934865fff => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:47.317807 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.318287 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.318399 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.338148 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:47.339840 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.401214 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1441/2853] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:47.411408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.411757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.412098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.432687 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:47.433893 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.442041 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1413/2854] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:47.446413 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.456014 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22640c4-20d3-4156-a29e-84afbb5fa000 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.460546 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.461131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.461409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.461680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.462374 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:35:47.462678 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.464271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.476272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1442/2855] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:35:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:35:47.480313 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.489806 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77e6153-6623-41d8-9140-d5cae2e837d7 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.494427 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.495014 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.495288 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.495557 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.496219 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.498123 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.530898 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1414/2856] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:35:47 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:35:47.532086 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.542358 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c6386d-e412-4e4a-ab76-6e74dcd717bb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.591494 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:47.603466 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.604165 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.604426 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.604725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.605349 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.607306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.628875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1443/2857] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:35:47 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:35:47.633329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.645218 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de10e2a-2a1f-4135-9780-0b3f08669d51 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.649857 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.650475 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.650752 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.651019 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.651645 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.653216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.663046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1415/2858] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:47.667019 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.677097 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb351f-4641-4c92-bad8-a69be0c7eefb admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.681725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ff217-16f5-4b9a-b3ac-8ae3fda58869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.682361 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ff217-16f5-4b9a-b3ac-8ae3fda58869 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.682641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ff217-16f5-4b9a-b3ac-8ae3fda58869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.682914 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ff217-16f5-4b9a-b3ac-8ae3fda58869 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.683552 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ff217-16f5-4b9a-b3ac-8ae3fda58869 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.685134 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ff217-16f5-4b9a-b3ac-8ae3fda58869 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.696263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1444/2859] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:35:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:35:47.700382 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.711158 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f5d8b2-70e8-4ad9-9879-154a3e6e3964 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.715814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.716456 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.716736 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.717192 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] PATH_INFO: `/v3/projects/a631538876834392b0060371f60d16ba/users/c81ccab1533247c491f928a3838983ec/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.731780 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a631538876834392b0060371f60d16ba, user_id=c81ccab1533247c491f928a3838983ec, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.735183 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.747735 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1416/2860] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:35:47 2022] PUT /identity/v3/projects/a631538876834392b0060371f60d16ba/users/c81ccab1533247c491f928a3838983ec/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:35:47.758592 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.758880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.759178 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.786290 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] MFA Rules not processed for user `c81ccab1533247c491f928a3838983ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:35:47.787497 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.849474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1445/2861] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:35:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:35:47.861790 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:35:47.863563 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:35:47.887049 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:47.942897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e3866b-5aa7-4ee1-9bb0-6c2f699940d8 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:35:47.946741 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992c8cc-08cf-4490-909f-1bce27d2e4d2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:35:47.947372 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992c8cc-08cf-4490-909f-1bce27d2e4d2 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:35:47.947654 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992c8cc-08cf-4490-909f-1bce27d2e4d2 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:35:47.947929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992c8cc-08cf-4490-909f-1bce27d2e4d2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:35:47.956355 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992c8cc-08cf-4490-909f-1bce27d2e4d2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:35:47.957760 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992c8cc-08cf-4490-909f-1bce27d2e4d2 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:35:47.966516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1417/2862] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:35:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:35:57.846404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:35:57.962480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=26 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:05.767439 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:05.828471 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509eaac3-580a-4527-9c00-1599d97d8b8e None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:05.832913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:05.833531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:05.833806 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:05.834075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] PATH_INFO: `/v3/users/c81ccab1533247c491f928a3838983ec` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:05.835210 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c81ccab1533247c491f928a3838983ec)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:05.837129 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:05.881175 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c81ccab1533247c491f928a3838983ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.881845 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c81ccab1533247c491f928a3838983ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.889463 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c81ccab1533247c491f928a3838983ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.890263 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c81ccab1533247c491f928a3838983ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.902442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c81ccab1533247c491f928a3838983ec'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:05.915683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1446/2863] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:05 2022] DELETE /identity/v3/users/c81ccab1533247c491f928a3838983ec => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:05.920214 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:05.920742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:05.920742 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:05.939351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:05.940534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:05.997216 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1418/2864] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:06.006970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.008369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.064431 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d23f07f-cee6-4d85-b0bd-3f672a2d6c8b admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.067923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fecad5-b94a-4a7e-8249-5fdc9d141ad1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.068517 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fecad5-b94a-4a7e-8249-5fdc9d141ad1 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.068795 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fecad5-b94a-4a7e-8249-5fdc9d141ad1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.069061 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fecad5-b94a-4a7e-8249-5fdc9d141ad1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.078614 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fecad5-b94a-4a7e-8249-5fdc9d141ad1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.080075 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fecad5-b94a-4a7e-8249-5fdc9d141ad1 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.089679 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1447/2865] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:06.349910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.351386 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.363777 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e35ae-1627-49cf-97c7-d784ad7bedbd None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.368860 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.369486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.369764 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.370032 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] PATH_INFO: `/v3/projects/a631538876834392b0060371f60d16ba` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.371184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a631538876834392b0060371f60d16ba)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.373113 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.418606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a631538876834392b0060371f60d16ba', 'request_id': 'req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.425390 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a631538876834392b0060371f60d16ba', 'request_id': 'req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.432761 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a631538876834392b0060371f60d16ba', 'request_id': 'req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.444246 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] The token cache is being invalidate because project a631538876834392b0060371f60d16ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:06.444537 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.445170 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-48f7fd6d-bfd6-4cb5-85ae-c7346515906c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:06.446699 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1419/2866] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:06 2022] DELETE /identity/v3/projects/a631538876834392b0060371f60d16ba => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:06.451572 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.451850 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.452120 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.471970 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:06.473424 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.490414 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:06.534924 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1448/2867] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:06.542680 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.542964 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.543240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.562619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:06.563810 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:06.572656 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1420/2868] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:06.577809 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.587441 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794e7d0-089c-4382-aa8b-d237933057bf None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.593268 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.593802 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.594088 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.594410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.595079 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:06.595366 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.596961 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.609045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1449/2869] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:06.613357 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.622599 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b99cef1-aaf6-4476-bc0f-5ec99303c052 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.627935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.628534 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.628815 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.629088 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.629733 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.631666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.660395 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1421/2870] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:06 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:06.664635 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.674512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6708be7b-cf1c-47d3-b611-2da33db2ac79 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.744475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.745101 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.745379 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.745651 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.746341 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.748257 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.772528 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1450/2871] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:06 2022] POST /identity/v3/users => generated 285 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:06.775933 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.787353 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c3fad6-0b50-4156-ae2d-2da73b881c83 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.791921 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.792541 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.792820 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.793097 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.793729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.795823 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.805516 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1422/2872] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:06.809235 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.819143 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05ad2a-3955-43a7-8d83-5361b214a349 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.823704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2a491a-ec4e-4189-8935-0b87540e3b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.824272 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a491a-ec4e-4189-8935-0b87540e3b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.824553 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a491a-ec4e-4189-8935-0b87540e3b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.824828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2a491a-ec4e-4189-8935-0b87540e3b60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.825520 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a491a-ec4e-4189-8935-0b87540e3b60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.827828 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2a491a-ec4e-4189-8935-0b87540e3b60 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.838473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1451/2873] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:06.843514 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:06.854081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f517c-32b0-4059-af1f-69e1e581f2bf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:06.860709 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:06.861351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.861633 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.861910 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] PATH_INFO: `/v3/projects/fb904023eba242459ddfe37cf8b5b997/users/309cd3924b9e445b95c812cbf43ecf55/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.878221 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb904023eba242459ddfe37cf8b5b997, user_id=309cd3924b9e445b95c812cbf43ecf55, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:06.880897 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:06.894449 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1423/2874] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:06 2022] PUT /identity/v3/projects/fb904023eba242459ddfe37cf8b5b997/users/309cd3924b9e445b95c812cbf43ecf55/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:06.905920 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:06.906222 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:06.906503 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:06.933244 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] MFA Rules not processed for user `309cd3924b9e445b95c812cbf43ecf55`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:06.934482 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:07.000191 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1452/2875] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:07.011658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:07.013408 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:07.034977 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:07.078721 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23407d69-fe5e-43e2-813b-28d91fa3cfc2 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:07.082348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae4f13e-6e7f-44f5-865f-4db1d25567d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5c6c66598c74c11a1c9cac119644c19', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:07.082943 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4f13e-6e7f-44f5-865f-4db1d25567d1 service glance] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:07.083217 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4f13e-6e7f-44f5-865f-4db1d25567d1 service glance] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:07.083486 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae4f13e-6e7f-44f5-865f-4db1d25567d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:07.093166 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4f13e-6e7f-44f5-865f-4db1d25567d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:07.094718 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae4f13e-6e7f-44f5-865f-4db1d25567d1 service glance] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:07.103814 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1424/2876] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:07 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:16.996702 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:17.099724 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=17 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:34.863648 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:34.926113 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db4f44-6832-4424-80e9-eb413de49e15 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:34.934377 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:34.935011 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:34.935284 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:34.935549 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] PATH_INFO: `/v3/users/309cd3924b9e445b95c812cbf43ecf55` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:34.937094 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=309cd3924b9e445b95c812cbf43ecf55)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:34.939195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:34.986538 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '309cd3924b9e445b95c812cbf43ecf55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.987187 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '309cd3924b9e445b95c812cbf43ecf55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.994604 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '309cd3924b9e445b95c812cbf43ecf55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:34.995234 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '309cd3924b9e445b95c812cbf43ecf55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.006857 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '309cd3924b9e445b95c812cbf43ecf55'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.022895 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1453/2877] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:34 2022] DELETE /identity/v3/users/309cd3924b9e445b95c812cbf43ecf55 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:35.029865 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.030551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.030551 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.052885 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:35.054098 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.118613 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1425/2878] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:35.129074 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.130525 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.189672 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed1d3d-0f88-4d21-9fbe-1a16ea9fae59 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.193362 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b49e66-dde1-4545-bb9c-84b5de045428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.193971 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b49e66-dde1-4545-bb9c-84b5de045428 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.194270 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b49e66-dde1-4545-bb9c-84b5de045428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.194541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b49e66-dde1-4545-bb9c-84b5de045428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.204634 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b49e66-dde1-4545-bb9c-84b5de045428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.206068 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b49e66-dde1-4545-bb9c-84b5de045428 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.216410 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1454/2879] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:35.518694 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.520265 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.533941 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034855b1-ac00-48f0-b1ce-dc31332262f2 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.538663 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.539289 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.539590 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.539859 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] PATH_INFO: `/v3/projects/fb904023eba242459ddfe37cf8b5b997` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.540965 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb904023eba242459ddfe37cf8b5b997)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.543201 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.603209 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb904023eba242459ddfe37cf8b5b997', 'request_id': 'req-04562157-41ae-4d6f-ae33-6d979c0f5304'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.611410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb904023eba242459ddfe37cf8b5b997', 'request_id': 'req-04562157-41ae-4d6f-ae33-6d979c0f5304'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.619642 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb904023eba242459ddfe37cf8b5b997', 'request_id': 'req-04562157-41ae-4d6f-ae33-6d979c0f5304'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.632944 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] The token cache is being invalidate because project fb904023eba242459ddfe37cf8b5b997 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:35.633666 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.634658 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-04562157-41ae-4d6f-ae33-6d979c0f5304 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:35.635713 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1426/2880] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:35 2022] DELETE /identity/v3/projects/fb904023eba242459ddfe37cf8b5b997 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:35.641040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.641314 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.641660 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.667409 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:35.668627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.690451 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:35.737443 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1455/2881] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:35.746381 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.746878 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.747329 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.769978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:35.771244 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:35.779696 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1427/2882] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:35.785095 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.794902 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be00b739-7eb4-43b4-bb69-8e8982f57d90 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.799756 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.800356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.800641 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.800913 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.801568 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:35.801875 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.803512 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.820237 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1456/2883] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:35.824564 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.834676 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2b37-8cfe-4d8f-ae68-bea8db94115e None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.839297 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.839889 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.840191 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.840460 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.841081 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.843038 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.870189 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1428/2884] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:35.874360 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.884427 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564890ea-7504-43b3-ba4f-6c0ac25684c1 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.940668 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.941308 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.941588 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.941863 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.942543 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.944477 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.966356 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1457/2885] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:35 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:35.971096 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:35.981438 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e69dd5d-85ba-476d-b640-0a74322859ed admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:35.986136 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:35.986757 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:35.987034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:35.987306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:35.987931 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:35.989500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:35.999198 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1429/2886] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:35 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:36.003658 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.013487 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f57fc1-94d4-4b7f-b42f-732a2e5b786e admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.018643 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a1ae-1d29-491a-8f66-d3e1f98d16c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.019228 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a1ae-1d29-491a-8f66-d3e1f98d16c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.019502 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a1ae-1d29-491a-8f66-d3e1f98d16c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.019772 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a1ae-1d29-491a-8f66-d3e1f98d16c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.020402 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a1ae-1d29-491a-8f66-d3e1f98d16c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.021989 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a1ae-1d29-491a-8f66-d3e1f98d16c0 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.033150 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1458/2887] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:36.037303 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.047787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64072dd2-22a8-40ec-a4cc-1a10c2546940 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.052399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.053040 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.053323 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.053602 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] PATH_INFO: `/v3/projects/1ecc7c5e515046388e9d73a91df37372/users/5235c2dab58f4f9285c3878591dade45/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.068484 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ecc7c5e515046388e9d73a91df37372, user_id=5235c2dab58f4f9285c3878591dade45, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.071291 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.085935 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1430/2888] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:36 2022] PUT /identity/v3/projects/1ecc7c5e515046388e9d73a91df37372/users/5235c2dab58f4f9285c3878591dade45/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:36.097842 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.098138 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.098473 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.127070 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] MFA Rules not processed for user `5235c2dab58f4f9285c3878591dade45`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:36.128275 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.191683 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1459/2889] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:36.203298 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.205034 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.226715 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.262903 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba07160-deb5-4bdb-bc69-9b4fca284acf admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.267238 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27591857-42be-4a0d-b0b0-4eb54ff2e030 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.267837 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591857-42be-4a0d-b0b0-4eb54ff2e030 service cinder] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.268116 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591857-42be-4a0d-b0b0-4eb54ff2e030 service cinder] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.268389 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27591857-42be-4a0d-b0b0-4eb54ff2e030 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.276555 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27591857-42be-4a0d-b0b0-4eb54ff2e030 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.277970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27591857-42be-4a0d-b0b0-4eb54ff2e030 service cinder] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.286961 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1431/2890] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:36.311627 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.331243 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:36.376626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce007215-9498-40c0-8f56-fd4c1764f3c4 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.381164 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.381784 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.382062 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.382395 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] PATH_INFO: `/v3/users/5235c2dab58f4f9285c3878591dade45` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.383526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5235c2dab58f4f9285c3878591dade45)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.385442 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.430987 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5235c2dab58f4f9285c3878591dade45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:36.431645 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5235c2dab58f4f9285c3878591dade45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:36.439169 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5235c2dab58f4f9285c3878591dade45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:36.439818 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5235c2dab58f4f9285c3878591dade45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:36.451346 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.notifications [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5235c2dab58f4f9285c3878591dade45'} {{(pid=81361) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:36.464631 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1460/2891] 10.222.0.13 () {58 vars in 1224 bytes} [Tue Oct 4 10:36:36 2022] DELETE /identity/v3/users/5235c2dab58f4f9285c3878591dade45 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:36.469264 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.469543 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.469817 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.489937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:36.491186 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.548807 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1432/2892] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:36.558529 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.559911 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.617344 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7ddbb9-4614-4e32-9b1e-6e6156b1159a admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.621008 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9f4b49-e8af-4298-87bc-8ce1979fcf69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.621612 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f4b49-e8af-4298-87bc-8ce1979fcf69 service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.621887 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f4b49-e8af-4298-87bc-8ce1979fcf69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.622155 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9f4b49-e8af-4298-87bc-8ce1979fcf69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.632103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f4b49-e8af-4298-87bc-8ce1979fcf69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.633533 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9f4b49-e8af-4298-87bc-8ce1979fcf69 service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.643018 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1461/2893] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:36.915916 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:36.917359 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:36.931731 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffd034-1149-4f0e-9e35-26ea28ab3210 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:36.936568 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:36.937262 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:36.937560 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:36.937799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] PATH_INFO: `/v3/projects/1ecc7c5e515046388e9d73a91df37372` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:36.938937 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ecc7c5e515046388e9d73a91df37372)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:36.940842 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:36.989301 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ecc7c5e515046388e9d73a91df37372', 'request_id': 'req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:36.994558 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ecc7c5e515046388e9d73a91df37372', 'request_id': 'req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:37.002073 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ecc7c5e515046388e9d73a91df37372', 'request_id': 'req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a'} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:37.012781 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] The token cache is being invalidate because project 1ecc7c5e515046388e9d73a91df37372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81362) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 04 10:36:37.013065 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:37.013678 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.notifications [None req-b11bb3f0-8eaa-4abb-afe2-0fac81352c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81362) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 04 10:36:37.015258 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1433/2894] 10.222.0.13 () {58 vars in 1233 bytes} [Tue Oct 4 10:36:36 2022] DELETE /identity/v3/projects/1ecc7c5e515046388e9d73a91df37372 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:37.020119 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.020587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.020710 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.040285 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:37.041475 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:37.074292 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:37.102613 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1462/2895] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:37.110526 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.110984 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.111103 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.131844 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:37.133033 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:37.141315 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1434/2896] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:37.145606 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.155131 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64af01bb-7b32-4e78-bb1d-366fb934ae27 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.159868 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.160461 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.160732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.160995 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.161632 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81361) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 04 10:36:37.161928 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.163537 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.175368 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1463/2897] 10.222.0.13 () {58 vars in 1153 bytes} [Tue Oct 4 10:36:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 04 10:36:37.179636 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.190616 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98186b38-cf99-45fd-8d7f-1adc5275e7bb None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.195259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.195845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.196120 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.196387 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.197003 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.198908 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.225867 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1435/2898] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:37 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:37.230036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.240663 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650be33a-8460-4545-adc0-9668944d02a8 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.295294 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.295894 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.296180 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.296923 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.297551 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.299489 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.320278 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1464/2899] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:37 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:37.324366 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.334409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0ec9dd-1c2c-4977-87ce-ba85485b9e4d admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.338850 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.339434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.339710 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.339980 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.340610 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.342179 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.353759 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1436/2900] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:37.357621 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.371774 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e70219-4098-4414-99b0-ec7b9d7ad32c admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.378259 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28ddae5-7959-4ac4-af96-bac3926ed26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.378856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ddae5-7959-4ac4-af96-bac3926ed26e admin admin] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.379135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ddae5-7959-4ac4-af96-bac3926ed26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.379407 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ddae5-7959-4ac4-af96-bac3926ed26e admin admin] PATH_INFO: `/v3/roles` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.380045 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ddae5-7959-4ac4-af96-bac3926ed26e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.382699 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28ddae5-7959-4ac4-af96-bac3926ed26e admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.399719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1465/2901] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:37 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:37.403832 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.414048 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800b512b-4e06-425a-843f-40f23b8b6b13 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.418606 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.419240 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] REQUEST_METHOD: `PUT` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.419513 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.419782 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] PATH_INFO: `/v3/projects/d2311599c33d4d6b99b3293c10c46383/users/9500747a774d4c2a89f7e0719156a797/roles/7c12d43981d3489c96be96ccaf741270` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.422787 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:37.437989 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2311599c33d4d6b99b3293c10c46383, user_id=9500747a774d4c2a89f7e0719156a797, role_id=7c12d43981d3489c96be96ccaf741270)` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.440729 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.453619 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1437/2902] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:37 2022] PUT /identity/v3/projects/d2311599c33d4d6b99b3293c10c46383/users/9500747a774d4c2a89f7e0719156a797/roles/7c12d43981d3489c96be96ccaf741270 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:37.459303 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb12e-cf3b-419e-bd58-4a5c3c9dabe5 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.459698 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb12e-cf3b-419e-bd58-4a5c3c9dabe5 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.459983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fb12e-cf3b-419e-bd58-4a5c3c9dabe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.479038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-ee5fb12e-cf3b-419e-bd58-4a5c3c9dabe5 None None] MFA Rules not processed for user `789f896e36c14ef9a5f6c8970e9b3e4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:37.480248 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-ee5fb12e-cf3b-419e-bd58-4a5c3c9dabe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:37.539157 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1466/2903] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:37.549561 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:37.551284 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:37.606193 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d481f3-4ea7-439c-829a-036abbfc3b3e admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:37.610365 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:37.610957 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:37.611231 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:37.611500 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:37.620409 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:37.622118 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:37.631452 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1438/2904] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:41.817844 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:41.818404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:41.818404 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:41.852509 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] MFA Rules not processed for user `9500747a774d4c2a89f7e0719156a797`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:41.853974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:41.892342 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:41.931933 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1467/2905] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:41.936399 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:41.948348 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d499cb-3c57-459f-a4d5-56dbcaa48418 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:41.953968 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:41.954623 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:41.954978 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:41.955259 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] PATH_INFO: `/v3/projects` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:41.955803 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:41.957743 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:41.991434 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1439/2906] 10.222.0.13 () {60 vars in 1153 bytes} [Tue Oct 4 10:36:41 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 04 10:36:41.995319 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:42.017205 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e97f5-1f7f-4dd5-a52d-eef3ae3c3139 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:42.085941 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:42.086667 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:42.086983 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:42.087253 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] PATH_INFO: `/v3/users` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:42.087890 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:42.090032 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:42.115587 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1468/2907] 10.222.0.13 () {60 vars in 1144 bytes} [Tue Oct 4 10:36:41 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 04 10:36:42.119430 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:42.133462 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb2d455-be0e-42cc-88e9-b6af4c7ab61a admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:42.139883 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:42.140799 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:42.141316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:42.141703 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:42.142480 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:42.144649 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:42.157464 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1440/2908] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:42.163547 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:42.175721 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1aede-89dd-4c92-ade0-08238f8c5850 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:42.180757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:42.181423 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:42.181701 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:42.181972 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] PATH_INFO: `/v3/projects/3b59124579f5436482f5e11302a0a60a/users/e8665e1b9aed419cb700b4393792e6f7/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:42.198843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b59124579f5436482f5e11302a0a60a, user_id=e8665e1b9aed419cb700b4393792e6f7, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:42.201718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:42.215917 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1469/2909] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:42 2022] PUT /identity/v3/projects/3b59124579f5436482f5e11302a0a60a/users/e8665e1b9aed419cb700b4393792e6f7/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:42.220472 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:42.230754 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95b5a4b-5988-472e-a102-b8a1ce73e192 admin admin] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:42.293469 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c874319-578c-426e-87a0-1dd5770ac9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:42.294101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c874319-578c-426e-87a0-1dd5770ac9cf admin admin] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:42.294411 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c874319-578c-426e-87a0-1dd5770ac9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:42.294688 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c874319-578c-426e-87a0-1dd5770ac9cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:42.295332 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c874319-578c-426e-87a0-1dd5770ac9cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:42.297863 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c874319-578c-426e-87a0-1dd5770ac9cf admin admin] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:42.309141 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1441/2910] 10.222.0.13 () {58 vars in 1122 bytes} [Tue Oct 4 10:36:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 04 10:36:42.313541 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:42.324282 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a13eaf-b2a9-4d67-ac40-6668cef48635 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:42.329209 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '789f896e36c14ef9a5f6c8970e9b3e4a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fe7a43104df8480199dd56c9a29fe00d', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:42.329978 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:42.330353 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:42.330694 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] PATH_INFO: `/v3/projects/3b59124579f5436482f5e11302a0a60a/users/e8665e1b9aed419cb700b4393792e6f7/roles/e8cffbbc9b164ec5b458727785767845` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:42.332725 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b59124579f5436482f5e11302a0a60a, user_id=e8665e1b9aed419cb700b4393792e6f7, role_id=e8cffbbc9b164ec5b458727785767845)` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:42.335531 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:42.348625 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1470/2911] 10.222.0.13 () {60 vars in 1483 bytes} [Tue Oct 4 10:36:42 2022] PUT /identity/v3/projects/3b59124579f5436482f5e11302a0a60a/users/e8665e1b9aed419cb700b4393792e6f7/roles/e8cffbbc9b164ec5b458727785767845 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 04 10:36:46.791101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13578d41-b5e2-441c-8276-5cc507ffaacd None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:46.791651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13578d41-b5e2-441c-8276-5cc507ffaacd None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:46.791651 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13578d41-b5e2-441c-8276-5cc507ffaacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:46.826503 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-13578d41-b5e2-441c-8276-5cc507ffaacd None None] MFA Rules not processed for user `e8665e1b9aed419cb700b4393792e6f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:36:46.828101 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-13578d41-b5e2-441c-8276-5cc507ffaacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:46.876083 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:46.911622 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1442/2912] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:36:46 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:36:46.931881 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:36:46.933669 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:36:47.025556 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66f0a6-7981-4f90-893b-b200d22bf1e2 admin admin] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:36:47.032005 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:36:47.032734 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:36:47.033116 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:36:47.033392 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:36:47.059271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:47.098605 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:36:47.100213 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:36:47.110666 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1471/2913] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:36:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:36:56.908481 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=12 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:36:57.105453 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=16 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:04.429872 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:04.430421 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:04.430535 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:04.449344 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] MFA Rules not processed for user `9500747a774d4c2a89f7e0719156a797`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:37:04.450569 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:04.457845 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1443/2914] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:37:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:37:04.500135 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:04.501526 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:04.558195 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2655ccc-c954-49f2-b82b-b31a99ae5059 service nova] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:04.562879 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99de0be14e684b3696a4570987119ea7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:04.563483 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:04.563763 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:04.564036 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:04.565791 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:04.579838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:04.581232 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:04.592401 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1472/2915] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:05.302417 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:05.303839 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:05.309202 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:05.310653 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:05.369790 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e350a60-45fd-4a93-8b87-396254d16eb4 service cinder] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:05.373271 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:05.374403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:05.374680 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:05.374950 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:05.376487 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc9ad33-a9e2-481a-8b7f-2e853b16d53b None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:05.377602 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:05.381350 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:05.381970 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:05.382270 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:05.382540 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:05.384188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:05.391974 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:05.393403 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:05.395963 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:05.397498 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:05.404513 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1473/2916] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 04 10:37:05.407881 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1444/2917] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:07.353104 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:07.363184 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21630a8-7609-48b2-8b5d-ad1c52bcc02b service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:07.367038 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:07.367732 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:07.372724 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:07.373035 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:07.375718 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:07.388821 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:07.390378 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:07.399622 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1474/2918] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:08.110939 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:08.112383 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:08.131455 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f298f65b-a37e-4f1b-969b-7b868b1d7001 service placement] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:08.138146 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:08.139711 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:08.139996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:08.140269 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:08.141996 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:08.156163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:08.157756 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:08.169023 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1445/2919] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:08 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:17.396182 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=58 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:18.164064 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=41 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:19.232720 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:19.247103 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c4aa2c-d5bd-4622-8aae-16565ee3a67e service neutron] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:19.252870 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a8514e96615c4785aefde8261120afe3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:19.253470 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:19.253749 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:19.254040 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:19.255803 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:37:19.268626 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:19.270224 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a4dbe13-f19e-4d1b-b213-1eec23276f4b service nova] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:19.281843 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1475/2920] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:22.108267 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:37:22.117572 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc779fa5-9450-464a-8484-d7f2123e8001 service nova] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:37:22.121188 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:37:22.121809 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:37:22.122109 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:37:22.122410 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:37:22.131163 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:37:22.132559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:37:22.141947 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1446/2921] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:37:22 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:37:29.276574 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:37:32.137592 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:25.115836 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] REQUEST_METHOD: `POST` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:25.116381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:25.116381 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:25.145986 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.auth.core [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] MFA Rules not processed for user `9500747a774d4c2a89f7e0719156a797`. Rule list: `[]` (Enabled: `True`). {{(pid=81361) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:39:25.147211 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:25.169462 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1476/2922] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:39:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 04 10:39:25.481929 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:25.517608 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f578dbbb-b74c-4a49-8406-76131c459ce7 service neutron] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:25.521184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:25.521771 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:25.522046 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:25.522339 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:25.524011 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:25.546099 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:25.547522 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c47bca1-3b6e-4c02-8644-31be8df7a1a8 service neutron] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:25.555351 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1447/2923] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:39:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:39:35.167692 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=15 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:35.551739 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=23 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:39:53.556369 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:39:53.557757 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:53.602245 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28d86e3-4595-401a-95b2-c37c3ea927b0 None None] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:39:53.607704 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:39:53.608306 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:39:53.608594 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:39:53.608869 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:39:53.610630 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.fernet_utils [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81361) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:39:53.623619 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:39:53.624856 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:39:53.635200 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1477/2924] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:39:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:40:03.630838 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=18 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:41:20.596326 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] REQUEST_METHOD: `POST` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:41:20.596919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:41:20.596919 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:41:20.617530 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.auth.core [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] MFA Rules not processed for user `9500747a774d4c2a89f7e0719156a797`. Rule list: `[]` (Enabled: `True`). {{(pid=81362) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 04 10:41:20.618683 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.fernet_utils [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81362) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 04 10:41:20.627316 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1448/2925] 10.222.0.13 () {58 vars in 958 bytes} [Tue Oct 4 10:41:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 04 10:41:30.611079 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=3 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:42:57.127880 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] Authenticating user token {{(pid=81361) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:42:57.137988 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84818a96-f2ba-4691-af62-bd5ccc13db7a service placement] Validating token access rules against request {{(pid=81361) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:42:57.141548 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e275f1a5-30a5-489d-bd53-7f5ee12a0dcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0e568115c67243d3904359b84e0a790e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81361) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:42:57.142133 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275f1a5-30a5-489d-bd53-7f5ee12a0dcb service neutron] REQUEST_METHOD: `GET` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:42:57.142448 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275f1a5-30a5-489d-bd53-7f5ee12a0dcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:42:57.142719 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e275f1a5-30a5-489d-bd53-7f5ee12a0dcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81361) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:42:57.150638 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275f1a5-30a5-489d-bd53-7f5ee12a0dcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:42:57.152067 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e275f1a5-30a5-489d-bd53-7f5ee12a0dcb service neutron] RBAC: Authorization granted {{(pid=81361) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:42:57.160105 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: [pid: 81361|app: 0|req: 1478/2926] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:42:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:43:07.156474 dv-cinder-860227-1-iscsi devstack@keystone.service[81361]: DEBUG dbcounter [-] [81361] Writing DB stats keystone:SELECT=6 {{(pid=81361) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 04 10:44:57.532192 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] Authenticating user token {{(pid=81362) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 04 10:44:57.544559 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3eb93-366c-432b-b769-3289fc462444 None None] Validating token access rules against request {{(pid=81362) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 04 10:44:57.549403 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f96f3d-d042-40be-86cf-4bb589074492 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c873c7e9014b46c8b1bf717f1e0fe514', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2e15ff0840284da2afb5e33d5453289e', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81362) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 04 10:44:57.549993 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f96f3d-d042-40be-86cf-4bb589074492 service placement] REQUEST_METHOD: `GET` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 04 10:44:57.550306 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f96f3d-d042-40be-86cf-4bb589074492 service placement] SCRIPT_NAME: `/identity` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 04 10:44:57.550582 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f96f3d-d042-40be-86cf-4bb589074492 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81362) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 04 10:44:57.558983 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f96f3d-d042-40be-86cf-4bb589074492 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 04 10:44:57.560544 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f96f3d-d042-40be-86cf-4bb589074492 service placement] RBAC: Authorization granted {{(pid=81362) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 04 10:44:57.570184 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: [pid: 81362|app: 0|req: 1449/2927] 10.222.0.13 () {60 vars in 1365 bytes} [Tue Oct 4 10:44:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 04 10:45:07.566015 dv-cinder-860227-1-iscsi devstack@keystone.service[81362]: DEBUG dbcounter [-] [81362] Writing DB stats keystone:SELECT=6 {{(pid=81362) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}